[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:54 dist-test-slave-07c2 krb5kdc[7195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:54 dist-test-slave-07c2 krb5kdc[7195](info): set up 2 sockets
Aug 11 06:22:54 dist-test-slave-07c2 krb5kdc[7195](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 06:22:55.189386 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:22:55 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:22:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:22:55.375839 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33683
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:55.739054 7209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:55.739519 7209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:55.739712 7209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:55.746925 7209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:22:55.747083 7209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:55.747210 7209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:55.747345 7209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:22:55.747416 7209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:22:55.757314 7209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:55.759057 7209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:55.760936 7209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:55.783320 7215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:55.784017 7214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:55.783984 7217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:55.784758 7209 server_base.cc:1034] running on GCE node
I20260811 06:22:55.785831 7209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:55.788038 7209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:55.789280 7209 hybrid_clock.cc:648] HybridClock initialized: now 1786429375789222 us; error 80 us; skew 500 ppm
I20260811 06:22:55.789795 7209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:55 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429375, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:55.797390 7209 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:22:55.799672 7209 webserver.cc:460] Webserver started at http://127.7.4.190:38637/ using document root <none> and password file <none>
I20260811 06:22:55.800719 7209 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:55.800829 7209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:55.801220 7209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:55.805006 7209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "c73200858eec44fb9c703ecc85358560"
format_stamp: "Formatted at 2026-08-11 06:22:55 on dist-test-slave-07c2"
server_key: "a508b403d2c027b7821a4876c4e90b5b"
server_key_iv: "1a526e3b58346cdd25f9236cbe4caec0"
server_key_version: "encryptionkey@0"
I20260811 06:22:55.805950 7209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "c73200858eec44fb9c703ecc85358560"
format_stamp: "Formatted at 2026-08-11 06:22:55 on dist-test-slave-07c2"
server_key: "a508b403d2c027b7821a4876c4e90b5b"
server_key_iv: "1a526e3b58346cdd25f9236cbe4caec0"
server_key_version: "encryptionkey@0"
I20260811 06:22:55.813449 7209 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 06:22:55.818578 7224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:55.820391 7209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 06:22:55.820681 7209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "c73200858eec44fb9c703ecc85358560"
format_stamp: "Formatted at 2026-08-11 06:22:55 on dist-test-slave-07c2"
server_key: "a508b403d2c027b7821a4876c4e90b5b"
server_key_iv: "1a526e3b58346cdd25f9236cbe4caec0"
server_key_version: "encryptionkey@0"
I20260811 06:22:55.820923 7209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:55.845309 7209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:55.850374 7209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:55.850746 7209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:55.873121 7209 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:33683
I20260811 06:22:55.873164 7280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:33683 every 8 connection(s)
I20260811 06:22:55.875223 7209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:22:55.881031 7281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:22:55.882776 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7209
I20260811 06:22:55.883142 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:22:55.885159 7186 external_mini_cluster.cc:1428] Setting key 8f229e29f8ea0d9da830625ceec32171
I20260811 06:22:55.895637 7281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560: Bootstrap starting.
I20260811 06:22:55.900545 7281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560: Neither blocks nor log segments found. Creating new log.
I20260811 06:22:55.902604 7281 log.cc:824] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560: Log is configured to *not* fsync() on all Append() calls
I20260811 06:22:55.906659 7281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560: No bootstrap required, opened a new log
I20260811 06:22:55.916786 7281 raft_consensus.cc:348] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } }
I20260811 06:22:55.917263 7281 raft_consensus.cc:374] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:22:55.917460 7281 raft_consensus.cc:729] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73200858eec44fb9c703ecc85358560, State: Initialized, Role: FOLLOWER
Aug 11 06:22:55 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:22:55.918751 7281 consensus_queue.cc:260] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } }
I20260811 06:22:55.919162 7281 raft_consensus.cc:388] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:22:55.919523 7281 raft_consensus.cc:482] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:22:55.919821 7281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:22:55.922328 7281 raft_consensus.cc:504] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } }
I20260811 06:22:55.923324 7281 leader_election.cc:302] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c73200858eec44fb9c703ecc85358560; no voters:
I20260811 06:22:55.924388 7281 leader_election.cc:288] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:22:55.924587 7288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:22:55.926095 7288 raft_consensus.cc:686] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [term 1 LEADER]: Becoming Leader. State: Replica: c73200858eec44fb9c703ecc85358560, State: Running, Role: LEADER
I20260811 06:22:55.927175 7288 consensus_queue.cc:237] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } }
I20260811 06:22:55.927799 7281 sys_catalog.cc:564] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:22:55.932056 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:55.890637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.7.4.190:33683)
0811 06:22:55.891897 (+ 1260us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:55.891917 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:55.893650 (+ 1733us) server_negotiation.cc:407] Connection header received
0811 06:22:55.903792 (+ 10142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:55.903809 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:55.904387 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:55.904856 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:55.907890 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:55.910244 (+ 2354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:55.911928 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:55.913013 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:55.913262 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:55.919717 (+ 6455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:55.919785 (+ 68us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:55.919818 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:55.919853 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:55.923070 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:55.924386 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:55.924402 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:55.924410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:55.924720 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:55.925514 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:55.925521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:55.925527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:55.926914 (+ 1387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:55.927980 (+ 1066us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:55.929020 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0811 06:22:55.929553 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":479,"threads_started":1}
I20260811 06:22:55.938881 7289 sys_catalog.cc:455] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c73200858eec44fb9c703ecc85358560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } } }
I20260811 06:22:55.938956 7290 sys_catalog.cc:455] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c73200858eec44fb9c703ecc85358560. Latest consensus state: current_term: 1 leader_uuid: "c73200858eec44fb9c703ecc85358560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c73200858eec44fb9c703ecc85358560" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33683 } } }
I20260811 06:22:55.939666 7289 sys_catalog.cc:458] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:55.939731 7290 sys_catalog.cc:458] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560 [sys.catalog]: This master's current role is: LEADER
I20260811 06:22:55.941843 7297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:22:55.949563 7297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:22:55.962172 7297 catalog_manager.cc:1347] Generated new cluster ID: 515371a84c064b9a8f6a44df58ba4c9b
I20260811 06:22:55.962302 7297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:22:55.971801 7297 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:22:55.973110 7297 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:22:55.980379 7297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c73200858eec44fb9c703ecc85358560: Generated new TSK 0
I20260811 06:22:55.981232 7297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:22:56.032598 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:56.386231 7312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:56.386533 7312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:56.386636 7312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:56.394379 7312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:56.394502 7312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:56.394625 7312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:22:56.405315 7312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:56.406623 7312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:56.408106 7312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:56.423712 7318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:56.423840 7321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:56.423700 7319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:56.425180 7312 server_base.cc:1034] running on GCE node
I20260811 06:22:56.425895 7312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:56.426926 7312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:56.428284 7312 hybrid_clock.cc:648] HybridClock initialized: now 1786429376428139 us; error 39 us; skew 500 ppm
I20260811 06:22:56.428650 7312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:56 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:56.432348 7312 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:22:56.433897 7312 webserver.cc:460] Webserver started at http://127.7.4.129:39923/ using document root <none> and password file <none>
I20260811 06:22:56.434643 7312 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:56.434805 7312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:56.435094 7312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:56.438359 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "52b0aa9ff30840a59cec191436290754"
format_stamp: "Formatted at 2026-08-11 06:22:56 on dist-test-slave-07c2"
server_key: "55d2e6b71ef21d1f1d1f012d7715738d"
server_key_iv: "94b9e52492a0ae5f01ce13a6b09707d1"
server_key_version: "encryptionkey@0"
I20260811 06:22:56.439205 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "52b0aa9ff30840a59cec191436290754"
format_stamp: "Formatted at 2026-08-11 06:22:56 on dist-test-slave-07c2"
server_key: "55d2e6b71ef21d1f1d1f012d7715738d"
server_key_iv: "94b9e52492a0ae5f01ce13a6b09707d1"
server_key_version: "encryptionkey@0"
I20260811 06:22:56.445159 7312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:22:56.448684 7328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:56.450379 7312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:22:56.450536 7312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "52b0aa9ff30840a59cec191436290754"
format_stamp: "Formatted at 2026-08-11 06:22:56 on dist-test-slave-07c2"
server_key: "55d2e6b71ef21d1f1d1f012d7715738d"
server_key_iv: "94b9e52492a0ae5f01ce13a6b09707d1"
server_key_version: "encryptionkey@0"
I20260811 06:22:56.450719 7312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:56.470173 7312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:56.475059 7312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:56.475446 7312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:56.476675 7312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:22:56.478266 7312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:22:56.478348 7312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:56.478554 7312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:22:56.478609 7312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:56.516362 7312 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:38585
I20260811 06:22:56.516394 7462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:38585 every 8 connection(s)
I20260811 06:22:56.517783 7312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:22:56.523521 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7312
I20260811 06:22:56.523810 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:22:56.524641 7186 external_mini_cluster.cc:1428] Setting key 7ff8cc9d34d8373537352b075d3f59a7
Aug 11 06:22:56 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:22:56.548210 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:56.521339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:42937 (local address 127.7.4.190:33683)
0811 06:22:56.521819 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:56.521824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:56.522819 (+ 995us) server_negotiation.cc:407] Connection header received
0811 06:22:56.525631 (+ 2812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:56.525637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:56.525739 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:56.525861 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:56.527748 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:56.528688 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:56.531678 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:56.532328 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:56.532447 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:56.538604 (+ 6157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:56.538614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:56.538622 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:56.538678 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:56.543254 (+ 4576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:56.544656 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:56.544661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:56.544663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:56.544802 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:56.545372 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:56.545376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:56.545378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:56.545671 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:56.545790 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:56.547185 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0811 06:22:56.547677 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":282,"threads_started":1}
I20260811 06:22:56.549117 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:56.521899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33683 (local address 127.7.4.129:42937)
0811 06:22:56.522631 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:56.522680 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:56.525287 (+ 2607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:56.526115 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:56.526123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:56.526680 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:56.527539 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:56.527579 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:56.528866 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:56.528871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:56.531463 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:56.531483 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:56.532603 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:56.532624 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:56.533022 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:56.534208 (+ 1186us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:56.534280 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:56.538236 (+ 3956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:56.543588 (+ 5352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:56.543602 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:56.543624 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:56.544468 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:56.544943 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:56.544951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:56.544957 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:56.545249 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:56.545817 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:56.545828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:56.546413 (+ 585us) client_negotiation.cc:769] Sending connection context
0811 06:22:56.546888 (+ 475us) client_negotiation.cc:240] Negotiation successful
0811 06:22:56.547649 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":285,"threads_started":1}
I20260811 06:22:56.551240 7463 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33683
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
I20260811 06:22:56.551874 7463 heartbeater.cc:464] Registering TS with master...
I20260811 06:22:56.552831 7463 heartbeater.cc:511] Master 127.7.4.190:33683 requested a full tablet report, sending...
I20260811 06:22:56.556903 7241 ts_manager.cc:194] Registered new tserver with Master: 52b0aa9ff30840a59cec191436290754 (127.7.4.129:38585)
I20260811 06:22:56.559122 7241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:42937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
W20260811 06:22:57.458348 7186 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.4.130: real 0.931s user 0.005s sys 0.014s
I20260811 06:22:57.459080 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:22:57.562830 7463 heartbeater.cc:503] Master 127.7.4.190:33683 was elected leader, sending a full tablet report...
W20260811 06:22:57.815119 7470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:57.815407 7470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:57.815629 7470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:57.822710 7470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:57.822824 7470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:57.823024 7470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:22:57.832624 7470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:57.833774 7470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:57.835088 7470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:57.848104 7476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:57.848237 7475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:57.848201 7478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:57.849320 7470 server_base.cc:1034] running on GCE node
I20260811 06:22:57.849937 7470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:57.850835 7470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:57.852032 7470 hybrid_clock.cc:648] HybridClock initialized: now 1786429377851989 us; error 35 us; skew 500 ppm
I20260811 06:22:57.852347 7470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:57 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429377, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:57.855780 7470 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:22:57.857303 7470 webserver.cc:460] Webserver started at http://127.7.4.130:39289/ using document root <none> and password file <none>
I20260811 06:22:57.857941 7470 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:57.858084 7470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:57.858446 7470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:57.861258 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "6ea9dea63e734c89971b89f369c30ff9"
format_stamp: "Formatted at 2026-08-11 06:22:57 on dist-test-slave-07c2"
server_key: "96d7080307975b3c8d8c5ee924c3f744"
server_key_iv: "dfc5f80cbbe2291a400f8c6987f1e6ea"
server_key_version: "encryptionkey@0"
I20260811 06:22:57.862022 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "6ea9dea63e734c89971b89f369c30ff9"
format_stamp: "Formatted at 2026-08-11 06:22:57 on dist-test-slave-07c2"
server_key: "96d7080307975b3c8d8c5ee924c3f744"
server_key_iv: "dfc5f80cbbe2291a400f8c6987f1e6ea"
server_key_version: "encryptionkey@0"
I20260811 06:22:57.867548 7470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:22:57.870745 7485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:57.871977 7470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:22:57.872150 7470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "6ea9dea63e734c89971b89f369c30ff9"
format_stamp: "Formatted at 2026-08-11 06:22:57 on dist-test-slave-07c2"
server_key: "96d7080307975b3c8d8c5ee924c3f744"
server_key_iv: "dfc5f80cbbe2291a400f8c6987f1e6ea"
server_key_version: "encryptionkey@0"
I20260811 06:22:57.872332 7470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:57.887986 7470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:57.891666 7470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:57.891940 7470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:57.892889 7470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:22:57.894318 7470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:22:57.894398 7470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:57.894578 7470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:22:57.894636 7470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:57.934083 7470 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:36131
I20260811 06:22:57.934165 7620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:36131 every 8 connection(s)
I20260811 06:22:57.935442 7470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:22:57.940791 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7470
I20260811 06:22:57.940953 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:22:57.941560 7186 external_mini_cluster.cc:1428] Setting key bcfd22292dbd7116a7a674c30ee9dd6e
Aug 11 06:22:57 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429377, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:22:57.957863 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:57.938002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:57565 (local address 127.7.4.190:33683)
0811 06:22:57.938561 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:57.938568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:57.939519 (+ 951us) server_negotiation.cc:407] Connection header received
0811 06:22:57.941403 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:57.941408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:57.941510 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:57.941677 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:57.943435 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:57.944319 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:57.945828 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:57.946262 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:57.946390 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:57.950437 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:57.950443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:57.950446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:57.950476 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:57.954238 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:57.955126 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:57.955134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:57.955141 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:57.955333 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:57.955756 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:57.955763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:57.955769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:57.956150 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:57.956305 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:57.956937 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 06:22:57.957313 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":290,"threads_started":1}
I20260811 06:22:57.958225 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:57.938574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33683 (local address 127.7.4.130:57565)
0811 06:22:57.939343 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:57.939398 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:57.941142 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:57.941907 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:57.941914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:57.942461 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:57.943222 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:57.943246 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:57.944491 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:57.944495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:57.945685 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:57.945698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:57.946480 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:57.946487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:57.946706 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:57.947610 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:57.947638 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:57.950197 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:57.954373 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:57.954380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:57.954390 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:57.954993 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:57.955449 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:57.955452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:57.955454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:57.955622 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:57.956270 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:57.956275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:57.956538 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 06:22:57.956799 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 06:22:57.957151 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":273,"threads_started":1}
I20260811 06:22:57.959702 7621 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33683
I20260811 06:22:57.959975 7621 heartbeater.cc:464] Registering TS with master...
I20260811 06:22:57.960721 7621 heartbeater.cc:511] Master 127.7.4.190:33683 requested a full tablet report, sending...
I20260811 06:22:57.962323 7241 ts_manager.cc:194] Registered new tserver with Master: 6ea9dea63e734c89971b89f369c30ff9 (127.7.4.130:36131)
I20260811 06:22:57.963124 7241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:57565
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:22:57.992282 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:22:58.329324 7627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:22:58.329649 7627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:22:58.329830 7627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:22:58.337103 7627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:22:58.337245 7627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:22:58.337455 7627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:22:58.347211 7627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:22:58.348562 7627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:22:58.350168 7627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:22:58.364216 7634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:58.364248 7637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:22:58.364312 7635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:22:58.365756 7627 server_base.cc:1034] running on GCE node
I20260811 06:22:58.366357 7627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:22:58.367352 7627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:22:58.368585 7627 hybrid_clock.cc:648] HybridClock initialized: now 1786429378368529 us; error 37 us; skew 500 ppm
I20260811 06:22:58.368927 7627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:22:58.372090 7627 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:22:58.373620 7627 webserver.cc:460] Webserver started at http://127.7.4.131:35027/ using document root <none> and password file <none>
I20260811 06:22:58.374330 7627 fs_manager.cc:359] Metadata directory not provided
I20260811 06:22:58.374482 7627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:22:58.374802 7627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:22:58.377868 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "3187e1d0fd8f410f844f286079542058"
format_stamp: "Formatted at 2026-08-11 06:22:58 on dist-test-slave-07c2"
server_key: "b77a6490832227ef86d16cdeaafc3918"
server_key_iv: "c33a4cc2c8e07ea0f792bfa92d08bb8e"
server_key_version: "encryptionkey@0"
I20260811 06:22:58.378655 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "3187e1d0fd8f410f844f286079542058"
format_stamp: "Formatted at 2026-08-11 06:22:58 on dist-test-slave-07c2"
server_key: "b77a6490832227ef86d16cdeaafc3918"
server_key_iv: "c33a4cc2c8e07ea0f792bfa92d08bb8e"
server_key_version: "encryptionkey@0"
I20260811 06:22:58.383682 7627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:22:58.386838 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:58.388168 7627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:22:58.388337 7627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "3187e1d0fd8f410f844f286079542058"
format_stamp: "Formatted at 2026-08-11 06:22:58 on dist-test-slave-07c2"
server_key: "b77a6490832227ef86d16cdeaafc3918"
server_key_iv: "c33a4cc2c8e07ea0f792bfa92d08bb8e"
server_key_version: "encryptionkey@0"
I20260811 06:22:58.388564 7627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:22:58.409261 7627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:22:58.413103 7627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:22:58.413383 7627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:22:58.414362 7627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:22:58.415716 7627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:22:58.415820 7627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:58.415974 7627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:22:58.416047 7627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:22:58.470312 7627 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:34861
I20260811 06:22:58.470403 7770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:34861 every 8 connection(s)
I20260811 06:22:58.471693 7627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:22:58.472939 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7627
I20260811 06:22:58.473111 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:22:58.473694 7186 external_mini_cluster.cc:1428] Setting key 9d504ebaa9080dc5acfb46f480d61332
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:22:58.492453 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.474612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:51737 (local address 127.7.4.190:33683)
0811 06:22:58.475478 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.475486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.477496 (+ 2010us) server_negotiation.cc:407] Connection header received
0811 06:22:58.478618 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.478622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.478687 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.478773 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.480304 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.481158 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.482665 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.483038 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.483128 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.486615 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.486620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.486623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.486655 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.489433 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.490434 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.490437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.490439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.490552 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.490906 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.490909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.490911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.491159 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.491279 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.491990 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.492192 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":307,"threads_started":1}
I20260811 06:22:58.493223 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.475227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33683 (local address 127.7.4.131:51737)
0811 06:22:58.476223 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 06:22:58.476277 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:22:58.478389 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:22:58.478948 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:22:58.478954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:22:58.479542 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:22:58.480150 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:58.480167 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.481303 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.481307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:58.482522 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:22:58.482534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.483229 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.483234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:22:58.483617 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.484253 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:22:58.484280 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:22:58.486440 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:22:58.489708 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:58.489717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:58.489731 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:58.490324 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:58.490641 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:22:58.490644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:22:58.490646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:22:58.490811 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:22:58.491261 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:22:58.491266 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:22:58.491628 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 06:22:58.491873 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 06:22:58.492215 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":434,"threads_started":1}
I20260811 06:22:58.494402 7771 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33683
I20260811 06:22:58.494694 7771 heartbeater.cc:464] Registering TS with master...
I20260811 06:22:58.495438 7771 heartbeater.cc:511] Master 127.7.4.190:33683 requested a full tablet report, sending...
I20260811 06:22:58.496842 7241 ts_manager.cc:194] Registered new tserver with Master: 3187e1d0fd8f410f844f286079542058 (127.7.4.131:34861)
I20260811 06:22:58.497550 7241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:51737
I20260811 06:22:58.502172 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:22:58.560501 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.544899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36986 (local address 127.7.4.129:38585)
0811 06:22:58.545557 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.545565 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.545602 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:22:58.545825 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.545833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.546055 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.546200 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.547812 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.548917 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.550192 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.550806 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.550905 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.554896 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.554908 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.554926 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.554969 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.558109 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.558729 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.558736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.558738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.558854 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.559204 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.559207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.559209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.559577 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.559804 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.560022 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.560224 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":327,"threads_started":1}
W20260811 06:22:58.561808 7362 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36986
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:22:58.614832 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.601613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.7.4.129:38585)
0811 06:22:58.601838 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.601843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.601954 (+ 111us) server_negotiation.cc:407] Connection header received
0811 06:22:58.602082 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.602086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.602152 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.602236 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.603584 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.604323 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.605635 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.606044 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.606144 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.610084 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.610089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.610092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.610124 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.612750 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.613336 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.613339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.613341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.613464 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.613777 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.613780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.613782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.614007 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.614195 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.614438 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.614598 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 06:22:58.616765 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7312
I20260811 06:22:58.629343 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.617757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.7.4.129:38585)
0811 06:22:58.617921 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.617925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.618303 (+ 378us) server_negotiation.cc:407] Connection header received
0811 06:22:58.618411 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.618415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.618465 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.618539 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.619698 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.620431 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.621729 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.622146 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.622233 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.624390 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.624395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.624397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.624425 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.626974 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.627621 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.627627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.627629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.627750 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.628128 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.628131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.628133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.628360 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.628524 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.628868 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.629067 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:22:58.630089 7457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:58.718591 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7312
I20260811 06:22:58.740823 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7470
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:22:58.757079 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.741839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.7.4.130:36131)
0811 06:22:58.742443 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.742450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.742477 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:22:58.742545 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.742551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.742773 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.742903 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.744271 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.745304 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.746559 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.747099 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.747197 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.751334 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.751355 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.751382 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.751416 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.754203 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.754820 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.754826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.754829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.754952 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.755566 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.755569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.755571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.755931 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.756154 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.756459 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.756678 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":326,"threads_started":1}
I20260811 06:22:58.758437 7615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:58.842638 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7470
I20260811 06:22:58.864464 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7627
Aug 11 06:22:58 dist-test-slave-07c2 krb5kdc[7195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:22:58.881604 7795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:22:58.865469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.7.4.131:34861)
0811 06:22:58.866148 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 06:22:58.866155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:22:58.866184 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:22:58.866243 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:22:58.866250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:22:58.866459 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:22:58.866634 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:22:58.868175 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.869236 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.870544 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:22:58.871137 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:22:58.871284 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:22:58.875700 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:22:58.875712 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:22:58.875727 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:22:58.875766 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:22:58.878705 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.879412 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.879420 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.879422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.879549 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:22:58.879957 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:22:58.879961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:22:58.879963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:22:58.880316 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:22:58.880641 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 06:22:58.880968 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 06:22:58.881262 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":346,"threads_started":1}
I20260811 06:22:58.882922 7766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:58.967556 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7627
I20260811 06:22:58.989552 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7209
I20260811 06:22:58.990768 7276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:22:59.099576 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7209
2026-08-11T06:22:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5121 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:22:59 dist-test-slave-07c2 krb5kdc[7801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:22:59 dist-test-slave-07c2 krb5kdc[7801](info): set up 2 sockets
Aug 11 06:22:59 dist-test-slave-07c2 krb5kdc[7801](info): commencing operation
krb5kdc: starting...
W20260811 06:23:00.250211 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.008s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:00 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:00.436177 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45459
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:00.751947 7814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:00.752208 7814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:00.752400 7814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:00.759652 7814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:00.759775 7814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:00.759868 7814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:00.759963 7814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:00.760025 7814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:00.769927 7814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:00.771245 7814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:00.772644 7814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:00.784315 7820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:00.784292 7821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:00.784258 7823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:00.785162 7814 server_base.cc:1034] running on GCE node
I20260811 06:23:00.785869 7814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:00.787154 7814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:00.788393 7814 hybrid_clock.cc:648] HybridClock initialized: now 1786429380788334 us; error 62 us; skew 500 ppm
I20260811 06:23:00.788754 7814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:00 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429380, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:00.791977 7814 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:00.793457 7814 webserver.cc:460] Webserver started at http://127.7.4.190:33051/ using document root <none> and password file <none>
I20260811 06:23:00.794150 7814 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:00.794296 7814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:00.794610 7814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:00.797574 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "872a0ed45f814447aff55a9fc6b34d9b"
format_stamp: "Formatted at 2026-08-11 06:23:00 on dist-test-slave-07c2"
server_key: "efcbb2e1878e8af8bb31f9a848605e45"
server_key_iv: "8038cd6a48829c3c61e96ba5c98f5036"
server_key_version: "encryptionkey@0"
I20260811 06:23:00.798316 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "872a0ed45f814447aff55a9fc6b34d9b"
format_stamp: "Formatted at 2026-08-11 06:23:00 on dist-test-slave-07c2"
server_key: "efcbb2e1878e8af8bb31f9a848605e45"
server_key_iv: "8038cd6a48829c3c61e96ba5c98f5036"
server_key_version: "encryptionkey@0"
I20260811 06:23:00.803428 7814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:23:00.806649 7831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:00.807961 7814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:00.808142 7814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "872a0ed45f814447aff55a9fc6b34d9b"
format_stamp: "Formatted at 2026-08-11 06:23:00 on dist-test-slave-07c2"
server_key: "efcbb2e1878e8af8bb31f9a848605e45"
server_key_iv: "8038cd6a48829c3c61e96ba5c98f5036"
server_key_version: "encryptionkey@0"
I20260811 06:23:00.808338 7814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:00.824242 7814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:00.827919 7814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:00.828192 7814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:00.846674 7814 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45459
I20260811 06:23:00.846755 7888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45459 every 8 connection(s)
I20260811 06:23:00.848102 7814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:00.851769 7889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:00.852366 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7814
I20260811 06:23:00.852571 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:00.853493 7186 external_mini_cluster.cc:1428] Setting key c5e198cbada4a0d2911bd382624a746f
I20260811 06:23:00.866568 7889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b: Bootstrap starting.
Aug 11 06:23:00 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:00.870227 7889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:00.871445 7889 log.cc:824] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:00.874161 7889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b: No bootstrap required, opened a new log
I20260811 06:23:00.877594 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:00.855653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.7.4.190:45459)
0811 06:23:00.856378 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:00.856389 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:00.856457 (+ 68us) server_negotiation.cc:407] Connection header received
0811 06:23:00.858861 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:00.858882 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:00.859266 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:00.859681 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:00.861614 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:00.862878 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:00.864672 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:00.865307 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:00.865442 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:00.870202 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:00.870209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:00.870225 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:00.870259 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:00.873470 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:00.874199 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:00.874204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:00.874210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:00.874424 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:00.874814 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:00.874819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:00.874823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:00.875391 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:00.875735 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:00.876259 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 06:23:00.876638 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":324,"threads_started":1}
I20260811 06:23:00.882395 7889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } }
I20260811 06:23:00.882787 7889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:00.882896 7889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872a0ed45f814447aff55a9fc6b34d9b, State: Initialized, Role: FOLLOWER
I20260811 06:23:00.883450 7889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } }
I20260811 06:23:00.883697 7889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:00.883884 7889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:00.884063 7889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:00.886348 7889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } }
I20260811 06:23:00.886817 7889 leader_election.cc:302] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 872a0ed45f814447aff55a9fc6b34d9b; no voters:
I20260811 06:23:00.887424 7889 leader_election.cc:288] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:00.887558 7894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:00.888469 7894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [term 1 LEADER]: Becoming Leader. State: Replica: 872a0ed45f814447aff55a9fc6b34d9b, State: Running, Role: LEADER
I20260811 06:23:00.889034 7894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } }
I20260811 06:23:00.889771 7889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:00.893108 7895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "872a0ed45f814447aff55a9fc6b34d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } } }
I20260811 06:23:00.893497 7895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:00.893982 7896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 872a0ed45f814447aff55a9fc6b34d9b. Latest consensus state: current_term: 1 leader_uuid: "872a0ed45f814447aff55a9fc6b34d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "872a0ed45f814447aff55a9fc6b34d9b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45459 } } }
I20260811 06:23:00.894331 7896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:00.894726 7905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:00.899262 7905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:00.906894 7905 catalog_manager.cc:1347] Generated new cluster ID: c18303294609411790152bfd0ff80a06
I20260811 06:23:00.906988 7905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:00.916833 7905 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:00.918547 7905 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:00.931365 7905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 872a0ed45f814447aff55a9fc6b34d9b: Generated new TSK 0
I20260811 06:23:00.932320 7905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:00.989357 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:01.316675 7917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:01.316988 7917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:01.317174 7917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:01.324358 7917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:01.324514 7917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:01.324692 7917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:01.334496 7917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:01.335806 7917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:01.337270 7917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:01.350080 7922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:01.350129 7923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:01.350163 7925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:01.351263 7917 server_base.cc:1034] running on GCE node
I20260811 06:23:01.351905 7917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:01.352762 7917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:01.353964 7917 hybrid_clock.cc:648] HybridClock initialized: now 1786429381353936 us; error 37 us; skew 500 ppm
I20260811 06:23:01.354245 7917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:01 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:01.357375 7917 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:01.358873 7917 webserver.cc:460] Webserver started at http://127.7.4.129:37039/ using document root <none> and password file <none>
I20260811 06:23:01.359586 7917 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:01.359740 7917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:01.360060 7917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:01.362927 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "6376821aa6334890bef213d9cd1e7946"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "258807dd69d94983ab2a991580a08ba1"
server_key_iv: "982329831d8ac8cc50b2e2233e84d9ee"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.363761 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "6376821aa6334890bef213d9cd1e7946"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "258807dd69d94983ab2a991580a08ba1"
server_key_iv: "982329831d8ac8cc50b2e2233e84d9ee"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.368753 7917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:23:01.371989 7933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.373288 7917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:01.373464 7917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "6376821aa6334890bef213d9cd1e7946"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "258807dd69d94983ab2a991580a08ba1"
server_key_iv: "982329831d8ac8cc50b2e2233e84d9ee"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.373656 7917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:01.391489 7917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:01.396312 7917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:01.396627 7917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:01.397617 7917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:01.398940 7917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:01.399047 7917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.399226 7917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:01.399329 7917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.433377 7917 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:41373
I20260811 06:23:01.433423 8068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:41373 every 8 connection(s)
I20260811 06:23:01.434707 7917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:01.438630 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 7917
I20260811 06:23:01.438853 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:01.439755 7186 external_mini_cluster.cc:1428] Setting key 0fa22df743f363a98100b33faa8aa18b
Aug 11 06:23:01 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:01.458592 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:01.437210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:52051 (local address 127.7.4.190:45459)
0811 06:23:01.437655 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:01.437661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:01.438989 (+ 1328us) server_negotiation.cc:407] Connection header received
0811 06:23:01.441311 (+ 2322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:01.441317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:01.441411 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:01.441537 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:01.443700 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.444387 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.447124 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.447789 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.447913 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:01.452100 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:01.452105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:01.452108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:01.452142 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:01.454800 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:01.455656 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:01.455664 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:01.455670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:01.455878 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:01.456363 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:01.456370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:01.456376 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:01.456798 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:01.456964 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:01.457786 (+ 822us) server_negotiation.cc:299] Negotiation successful
0811 06:23:01.458065 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":205,"threads_started":1}
I20260811 06:23:01.459142 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:01.437826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45459 (local address 127.7.4.129:52051)
0811 06:23:01.438738 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:01.438805 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:01.441050 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:01.441858 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:01.441871 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:01.442653 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:01.443499 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:01.443529 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.444582 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.444590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:01.446957 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:01.446975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.448030 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.448039 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:01.448379 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:01.449201 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:01.449243 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:01.451865 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:01.454946 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:01.454955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:01.454973 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:01.455500 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:01.456022 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:01.456025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:01.456028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:01.456228 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:01.456927 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:01.456934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:01.457355 (+ 421us) client_negotiation.cc:769] Sending connection context
0811 06:23:01.457603 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 06:23:01.457988 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":343,"threads_started":1}
I20260811 06:23:01.460785 8069 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45459
I20260811 06:23:01.461105 8069 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
I20260811 06:23:01.461962 8069 heartbeater.cc:511] Master 127.7.4.190:45459 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
I20260811 06:23:01.464541 7849 ts_manager.cc:194] Registered new tserver with Master: 6376821aa6334890bef213d9cd1e7946 (127.7.4.129:41373)
I20260811 06:23:01.466668 7849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:52051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:01.490092 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:01.824522 8076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:01.824808 8076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:01.824908 8076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:01.832289 8076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:01.832396 8076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:01.832509 8076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:01.842182 8076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:01.843454 8076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:01.844712 8076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:01.859701 8083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:01.860092 8082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:01.860553 8076 server_base.cc:1034] running on GCE node
W20260811 06:23:01.863394 8085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:01.864228 8076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:01.865381 8076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:01.866609 8076 hybrid_clock.cc:648] HybridClock initialized: now 1786429381866569 us; error 34 us; skew 500 ppm
I20260811 06:23:01.866950 8076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:01 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:01.870221 8076 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:01.871922 8076 webserver.cc:460] Webserver started at http://127.7.4.130:43931/ using document root <none> and password file <none>
I20260811 06:23:01.872639 8076 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:01.872776 8076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:01.873099 8076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:01.876271 8076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "ab0e086dd8e6448d92617ad5d90826e6"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "72afcd36d4a7c620a98bddc230e24eb9"
server_key_iv: "32206254f1325d5cae24cf3dd0f31009"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.877133 8076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "ab0e086dd8e6448d92617ad5d90826e6"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "72afcd36d4a7c620a98bddc230e24eb9"
server_key_iv: "32206254f1325d5cae24cf3dd0f31009"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.882378 8076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:23:01.886035 8093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.887461 8076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:23:01.887647 8076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "ab0e086dd8e6448d92617ad5d90826e6"
format_stamp: "Formatted at 2026-08-11 06:23:01 on dist-test-slave-07c2"
server_key: "72afcd36d4a7c620a98bddc230e24eb9"
server_key_iv: "32206254f1325d5cae24cf3dd0f31009"
server_key_version: "encryptionkey@0"
I20260811 06:23:01.887867 8076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:01.914072 8076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:01.918152 8076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:01.918448 8076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:01.919479 8076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:01.920847 8076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:01.920954 8076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.921110 8076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:01.921186 8076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:01.955698 8076 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:44269
I20260811 06:23:01.955745 8222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:44269 every 8 connection(s)
I20260811 06:23:01.956975 8076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:01.960568 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8076
I20260811 06:23:01.960803 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:01.961395 7186 external_mini_cluster.cc:1428] Setting key 5885e71cfe8dec0a83a1f7e81ac86493
Aug 11 06:23:01 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:01.977840 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:01.959679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:32913 (local address 127.7.4.190:45459)
0811 06:23:01.960148 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:01.960152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:01.961373 (+ 1221us) server_negotiation.cc:407] Connection header received
0811 06:23:01.963311 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:01.963317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:01.963410 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:01.963532 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:01.965055 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.965810 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.967406 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.967838 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.967976 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:01.971795 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:01.971800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:01.971804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:01.971835 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:01.974342 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:01.975340 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:01.975345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:01.975349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:01.975479 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:01.975961 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:01.975964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:01.975966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:01.976233 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:01.976354 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:01.977227 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 06:23:01.977496 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260811 06:23:01.979416 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:01.960328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45459 (local address 127.7.4.130:32913)
0811 06:23:01.961207 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:01.961245 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:01.963013 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:01.963787 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:01.963795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:01.964318 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:01.964880 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:01.964896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.965964 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.965970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:01.967188 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:01.967201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:01.968082 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:01.968090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:01.968325 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:01.969033 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:01.969056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:01.971574 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:01.974495 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:01.974507 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:01.974525 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:01.975180 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:01.975591 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:01.975599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:01.975605 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:01.975851 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:01.976387 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:01.976397 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:01.976850 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 06:23:01.977160 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 06:23:01.977787 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":379,"threads_started":1}
I20260811 06:23:01.980997 8223 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45459
I20260811 06:23:01.981312 8223 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:01.982036 8223 heartbeater.cc:511] Master 127.7.4.190:45459 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
I20260811 06:23:01.983613 7849 ts_manager.cc:194] Registered new tserver with Master: ab0e086dd8e6448d92617ad5d90826e6 (127.7.4.130:44269)
I20260811 06:23:01.984416 7849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:32913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:02.010947 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:02.347908 8231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:02.348197 8231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:02.348376 8231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:02.355906 8231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:02.356026 8231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:02.356241 8231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:02.366326 8231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:02.367766 8231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:02.369187 8231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:02.383381 8236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:02.383237 8237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:02.383267 8239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:02.384169 8231 server_base.cc:1034] running on GCE node
I20260811 06:23:02.384881 8231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:02.385869 8231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:02.387136 8231 hybrid_clock.cc:648] HybridClock initialized: now 1786429382387070 us; error 59 us; skew 500 ppm
I20260811 06:23:02.387509 8231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:02 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:02.390739 8231 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:02.392676 8231 webserver.cc:460] Webserver started at http://127.7.4.131:44673/ using document root <none> and password file <none>
I20260811 06:23:02.393419 8231 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:02.393573 8231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:02.393915 8231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:02.397040 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "6b296fbae0f84096860bff46d33b4abe"
format_stamp: "Formatted at 2026-08-11 06:23:02 on dist-test-slave-07c2"
server_key: "cbb4881d027a01827157a690f04ecdb1"
server_key_iv: "db533e2e95013939e0b9fc3c6a2e8754"
server_key_version: "encryptionkey@0"
I20260811 06:23:02.397859 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "6b296fbae0f84096860bff46d33b4abe"
format_stamp: "Formatted at 2026-08-11 06:23:02 on dist-test-slave-07c2"
server_key: "cbb4881d027a01827157a690f04ecdb1"
server_key_iv: "db533e2e95013939e0b9fc3c6a2e8754"
server_key_version: "encryptionkey@0"
I20260811 06:23:02.403129 8231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:23:02.406775 8246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:02.408224 8231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:02.408406 8231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "6b296fbae0f84096860bff46d33b4abe"
format_stamp: "Formatted at 2026-08-11 06:23:02 on dist-test-slave-07c2"
server_key: "cbb4881d027a01827157a690f04ecdb1"
server_key_iv: "db533e2e95013939e0b9fc3c6a2e8754"
server_key_version: "encryptionkey@0"
I20260811 06:23:02.408638 8231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:02.443919 8231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:02.447989 8231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:02.448295 8231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:02.449294 8231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:02.450637 8231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:02.450752 8231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:02.450911 8231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:02.450985 8231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:02.469787 8069 heartbeater.cc:503] Master 127.7.4.190:45459 was elected leader, sending a full tablet report...
I20260811 06:23:02.486433 8231 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:37681
I20260811 06:23:02.486510 8366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:37681 every 8 connection(s)
I20260811 06:23:02.487883 8231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:02.492257 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8231
I20260811 06:23:02.492439 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:02.492981 7186 external_mini_cluster.cc:1428] Setting key e19ea23728502ba85b7d8cbada64e79b
Aug 11 06:23:02 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:02.508591 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.490626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:51589 (local address 127.7.4.190:45459)
0811 06:23:02.491120 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.491124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.492187 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 06:23:02.494651 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.494657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.494754 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.494876 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:02.496690 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.497474 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.499049 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.499527 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.499666 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.502956 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:02.502961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:02.502964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:02.502995 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:02.505505 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:02.506214 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:02.506220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:02.506226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:02.506350 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:02.506781 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:02.506786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:02.506790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:02.507076 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:02.507256 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.507961 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.508192 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":245,"threads_started":1}
I20260811 06:23:02.509485 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.491228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45459 (local address 127.7.4.131:51589)
0811 06:23:02.492032 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.492077 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.494360 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.495132 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.495144 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.495962 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:02.496520 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.496536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.497640 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.497645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.498871 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.498882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.499773 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.499780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.500007 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.500621 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:02.500658 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:02.502749 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:02.505659 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:02.505668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:02.505682 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:02.506091 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:02.506466 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:02.506471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:02.506475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:02.506666 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:02.507217 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:02.507225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:02.507583 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.507863 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.508367 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":352,"threads_started":1}
I20260811 06:23:02.510921 8367 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45459
I20260811 06:23:02.511214 8367 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:02.512035 8367 heartbeater.cc:511] Master 127.7.4.190:45459 requested a full tablet report, sending...
I20260811 06:23:02.513684 7849 ts_manager.cc:194] Registered new tserver with Master: 6b296fbae0f84096860bff46d33b4abe (127.7.4.131:37681)
I20260811 06:23:02.514521 7849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:51589
I20260811 06:23:02.520620 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:23:02.540951 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.527808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.7.4.190:45459)
0811 06:23:02.528021 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.528030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.528597 (+ 567us) server_negotiation.cc:407] Connection header received
0811 06:23:02.528742 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.528746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.528812 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.528896 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:02.530487 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.531227 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.532509 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.532892 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.532985 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.535661 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:02.535666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:02.535669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:02.535702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:02.538316 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:02.539001 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:02.539005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:02.539007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:02.539147 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:02.539523 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:02.539526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:02.539528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:02.539864 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:02.540053 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.540374 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.540556 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:02.550025 7849 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:02.557482 7849 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:02.584105 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.573470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.7.4.130:44269)
0811 06:23:02.574124 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.574138 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.574291 (+ 153us) server_negotiation.cc:407] Connection header received
0811 06:23:02.574697 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.574709 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.574967 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.576082 (+ 1115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:02.576320 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.578104 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.580897 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.582580 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.582760 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.583015 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.583493 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.583698 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":357,"threads_started":1}
I20260811 06:23:02.584160 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.573280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:44269 (local address 127.0.0.1:51208)
0811 06:23:02.574133 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.574189 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.574435 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.575272 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.575283 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.575359 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:02.575872 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.575885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.578407 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.578415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.580710 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.580724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.582869 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.582876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.583130 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.583239 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.583415 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.583658 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":3,"thread_start_us":415,"threads_started":1}
I20260811 06:23:02.585239 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.574601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:41373 (local address 127.0.0.1:33694)
0811 06:23:02.575273 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.575324 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.575451 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.577873 (+ 2422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.577883 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.577915 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:02.578467 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.578486 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.580545 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.580551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.582313 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.582329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.584407 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.584413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.584646 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.584681 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.584747 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.584882 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":384,"threads_started":1}
I20260811 06:23:02.586388 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.574937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.7.4.129:41373)
0811 06:23:02.577183 (+ 2246us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.577191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.577223 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:23:02.577326 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.577336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.577628 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.578698 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:02.579014 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.580396 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.582536 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.584153 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.585531 (+ 1378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.585804 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.585876 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.586056 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2013,"thread_start_us":822,"threads_started":1}
I20260811 06:23:02.588528 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.576806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:37681 (local address 127.0.0.1:52544)
0811 06:23:02.577402 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.577419 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.577548 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.578491 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.578494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.578525 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:02.579877 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.579898 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.582625 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.582630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.585651 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.585665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.587701 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.587707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.587874 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.587912 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.588028 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.588159 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":58,"thread_start_us":275,"threads_started":1}
I20260811 06:23:02.588814 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.576102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.7.4.131:37681)
0811 06:23:02.576880 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.576888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.577525 (+ 637us) server_negotiation.cc:407] Connection header received
0811 06:23:02.577737 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.577744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.577926 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.578080 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:02.580119 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.581781 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.585844 (+ 4063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.587489 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.587646 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.587874 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.588082 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.588242 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":386,"threads_started":1}
I20260811 06:23:02.589301 8147 tablet_service.cc:1467] Processing CreateTablet for tablet f174188e3d7d49638cdbdb3a34cada7c (DEFAULT_TABLE table=test-table [id=cfe9db907d974d42aaf4dc658836908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:02.590189 7989 tablet_service.cc:1467] Processing CreateTablet for tablet f174188e3d7d49638cdbdb3a34cada7c (DEFAULT_TABLE table=test-table [id=cfe9db907d974d42aaf4dc658836908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:02.591172 8147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f174188e3d7d49638cdbdb3a34cada7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:02.591928 7989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f174188e3d7d49638cdbdb3a34cada7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:02.592067 8295 tablet_service.cc:1467] Processing CreateTablet for tablet f174188e3d7d49638cdbdb3a34cada7c (DEFAULT_TABLE table=test-table [id=cfe9db907d974d42aaf4dc658836908a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:02.593477 8295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f174188e3d7d49638cdbdb3a34cada7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:02.606616 8389 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Bootstrap starting.
I20260811 06:23:02.610284 8389 tablet_bootstrap.cc:654] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:02.611275 8391 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Bootstrap starting.
I20260811 06:23:02.612341 8389 log.cc:824] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:02.612385 8390 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Bootstrap starting.
I20260811 06:23:02.617024 8391 tablet_bootstrap.cc:654] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:02.618000 8390 tablet_bootstrap.cc:654] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:02.618721 8391 log.cc:824] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:02.619482 8390 log.cc:824] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:02.621999 8389 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: No bootstrap required, opened a new log
I20260811 06:23:02.622318 8389 ts_tablet_manager.cc:1397] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260811 06:23:02.622745 8390 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: No bootstrap required, opened a new log
I20260811 06:23:02.622829 8391 tablet_bootstrap.cc:492] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: No bootstrap required, opened a new log
I20260811 06:23:02.623006 8390 ts_tablet_manager.cc:1397] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 06:23:02.623060 8391 ts_tablet_manager.cc:1397] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260811 06:23:02.630971 8389 raft_consensus.cc:348] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.631392 8389 raft_consensus.cc:374] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:02.631494 8389 raft_consensus.cc:729] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b296fbae0f84096860bff46d33b4abe, State: Initialized, Role: FOLLOWER
I20260811 06:23:02.632120 8389 consensus_queue.cc:260] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.633395 8367 heartbeater.cc:503] Master 127.7.4.190:45459 was elected leader, sending a full tablet report...
I20260811 06:23:02.633895 8389 ts_tablet_manager.cc:1428] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 06:23:02.634974 8390 raft_consensus.cc:348] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.634917 8391 raft_consensus.cc:348] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.635398 8390 raft_consensus.cc:374] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:02.635439 8391 raft_consensus.cc:374] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:02.635532 8390 raft_consensus.cc:729] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6376821aa6334890bef213d9cd1e7946, State: Initialized, Role: FOLLOWER
I20260811 06:23:02.635551 8391 raft_consensus.cc:729] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0e086dd8e6448d92617ad5d90826e6, State: Initialized, Role: FOLLOWER
I20260811 06:23:02.636138 8391 consensus_queue.cc:260] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.636255 8390 consensus_queue.cc:260] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.637413 8223 heartbeater.cc:503] Master 127.7.4.190:45459 was elected leader, sending a full tablet report...
I20260811 06:23:02.639266 8390 ts_tablet_manager.cc:1428] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260811 06:23:02.639535 8391 ts_tablet_manager.cc:1428] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260811 06:23:02.687222 8070 tablet.cc:2369] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:02.709139 8224 tablet.cc:2369] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:02.739737 8368 tablet.cc:2369] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:02.921141 8396 raft_consensus.cc:482] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:02.921538 8396 raft_consensus.cc:504] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.923722 8396 leader_election.cc:288] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6376821aa6334890bef213d9cd1e7946 (127.7.4.129:41373), 6b296fbae0f84096860bff46d33b4abe (127.7.4.131:37681)
I20260811 06:23:02.931596 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.924162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:42187 (local address 127.7.4.129:41373)
0811 06:23:02.924426 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.924433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.925118 (+ 685us) server_negotiation.cc:407] Connection header received
0811 06:23:02.925397 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.925405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.925527 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.925656 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:02.926399 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.927377 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.929750 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.930591 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.930704 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.930841 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.931179 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.931346 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:23:02.931739 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.924147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:41373 (local address 127.7.4.130:42187)
0811 06:23:02.924998 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.925018 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.925248 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.925705 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.925710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.925745 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:02.926232 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.926247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.927501 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.927506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.929532 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.929544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.930857 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.930864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.931018 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.931042 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.931121 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.931208 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":21,"thread_start_us":500,"threads_started":1}
I20260811 06:23:02.931742 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.925342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:37753 (local address 127.7.4.131:37681)
0811 06:23:02.925500 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:02.925504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:02.926063 (+ 559us) server_negotiation.cc:407] Connection header received
0811 06:23:02.926175 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:02.926181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:02.926260 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:02.926351 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:02.926909 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.927760 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.929756 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.930625 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.930784 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.930895 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:02.931349 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 06:23:02.931482 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 06:23:02.932220 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:02.925010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:37681 (local address 127.7.4.130:37753)
0811 06:23:02.925946 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:02.925963 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:02.926061 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:02.926388 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:02.926392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:02.926415 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:02.926762 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.926773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.927901 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.927924 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.929578 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:02.929591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:02.930929 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:02.930935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:02.931149 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:02.931181 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:23:02.931274 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 06:23:02.931411 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":25,"thread_start_us":623,"threads_started":1}
I20260811 06:23:02.932924 8017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f174188e3d7d49638cdbdb3a34cada7c" candidate_uuid: "ab0e086dd8e6448d92617ad5d90826e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6376821aa6334890bef213d9cd1e7946" is_pre_election: true
I20260811 06:23:02.933367 8317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f174188e3d7d49638cdbdb3a34cada7c" candidate_uuid: "ab0e086dd8e6448d92617ad5d90826e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b296fbae0f84096860bff46d33b4abe" is_pre_election: true
I20260811 06:23:02.933701 8017 raft_consensus.cc:2454] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab0e086dd8e6448d92617ad5d90826e6 in term 0.
I20260811 06:23:02.933954 8317 raft_consensus.cc:2454] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab0e086dd8e6448d92617ad5d90826e6 in term 0.
I20260811 06:23:02.934546 8096 leader_election.cc:302] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6376821aa6334890bef213d9cd1e7946, ab0e086dd8e6448d92617ad5d90826e6; no voters:
I20260811 06:23:02.934930 8396 raft_consensus.cc:2781] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:02.935091 8396 raft_consensus.cc:482] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:02.935222 8396 raft_consensus.cc:3037] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:02.937477 8396 raft_consensus.cc:504] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.938048 8396 leader_election.cc:288] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [CANDIDATE]: Term 1 election: Requested vote from peers 6376821aa6334890bef213d9cd1e7946 (127.7.4.129:41373), 6b296fbae0f84096860bff46d33b4abe (127.7.4.131:37681)
I20260811 06:23:02.938597 8017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f174188e3d7d49638cdbdb3a34cada7c" candidate_uuid: "ab0e086dd8e6448d92617ad5d90826e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6376821aa6334890bef213d9cd1e7946"
I20260811 06:23:02.938664 8317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f174188e3d7d49638cdbdb3a34cada7c" candidate_uuid: "ab0e086dd8e6448d92617ad5d90826e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b296fbae0f84096860bff46d33b4abe"
I20260811 06:23:02.938802 8017 raft_consensus.cc:3037] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:02.938903 8317 raft_consensus.cc:3037] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:02.941602 8017 raft_consensus.cc:2454] T f174188e3d7d49638cdbdb3a34cada7c P 6376821aa6334890bef213d9cd1e7946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab0e086dd8e6448d92617ad5d90826e6 in term 1.
I20260811 06:23:02.941661 8317 raft_consensus.cc:2454] T f174188e3d7d49638cdbdb3a34cada7c P 6b296fbae0f84096860bff46d33b4abe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab0e086dd8e6448d92617ad5d90826e6 in term 1.
I20260811 06:23:02.942011 8096 leader_election.cc:302] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6376821aa6334890bef213d9cd1e7946, ab0e086dd8e6448d92617ad5d90826e6; no voters:
I20260811 06:23:02.942276 8396 raft_consensus.cc:2781] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:02.942721 8396 raft_consensus.cc:686] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [term 1 LEADER]: Becoming Leader. State: Replica: ab0e086dd8e6448d92617ad5d90826e6, State: Running, Role: LEADER
I20260811 06:23:02.943161 8396 consensus_queue.cc:237] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } }
I20260811 06:23:02.948195 7848 catalog_manager.cc:5495] T f174188e3d7d49638cdbdb3a34cada7c P ab0e086dd8e6448d92617ad5d90826e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab0e086dd8e6448d92617ad5d90826e6 (127.7.4.130). New cstate: current_term: 1 leader_uuid: "ab0e086dd8e6448d92617ad5d90826e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0e086dd8e6448d92617ad5d90826e6" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 44269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6376821aa6334890bef213d9cd1e7946" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 41373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b296fbae0f84096860bff46d33b4abe" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 37681 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:03.046134 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.031400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.7.4.129:41373)
0811 06:23:03.031650 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.031660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.032227 (+ 567us) server_negotiation.cc:407] Connection header received
0811 06:23:03.032374 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.032379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.032459 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.032540 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.034165 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.034909 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.036120 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.036520 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.036608 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.040614 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.040640 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.040657 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.040689 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.043692 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.044279 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.044286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.044289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.044419 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.044769 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.044772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.044774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.045107 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.045344 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.045570 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.045767 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:03.099241 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.085234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.7.4.129:41373)
0811 06:23:03.085487 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.085493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.085863 (+ 370us) server_negotiation.cc:407] Connection header received
0811 06:23:03.086011 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.086016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.086085 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.086175 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.087494 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.088280 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.089521 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.090041 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.090137 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.094271 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.094276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.094279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.094315 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.096888 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.097522 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.097526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.097528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.097691 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.098026 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.098029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.098031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.098291 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.098618 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.098714 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.098870 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:03.151058 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.136447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.7.4.129:41373)
0811 06:23:03.136679 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.136687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.136984 (+ 297us) server_negotiation.cc:407] Connection header received
0811 06:23:03.137250 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.137254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.137321 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.137426 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.138699 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.139440 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.140862 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.141306 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.141435 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.145413 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.145418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.145421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.145456 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.148237 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.148944 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.148951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.148955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.149111 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.149505 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.149510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.149514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.149799 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.150070 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.150384 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.150613 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:23:03.153067 7966 tablet_service.cc:2908] Rejecting scan request for tablet f174188e3d7d49638cdbdb3a34cada7c: Uninitialized: clean time has not yet been initialized
I20260811 06:23:03.157179 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7917
I20260811 06:23:03.169222 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.158066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.7.4.129:41373)
0811 06:23:03.158225 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.158229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.158607 (+ 378us) server_negotiation.cc:407] Connection header received
0811 06:23:03.158748 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.158751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.158818 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.158893 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.160076 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.160718 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.161927 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.162326 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.162422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.164645 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.164649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.164652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.164681 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.166996 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.167640 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.167643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.167645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.167753 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.168060 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.168063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.168065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.168316 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.168654 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.168808 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.168955 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:23:03.169853 8064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:03.259248 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7917
I20260811 06:23:03.283099 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8076
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:03.298239 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.283940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.7.4.130:44269)
0811 06:23:03.284456 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.284460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.284474 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:23:03.284529 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.284533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.284592 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.284722 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.285974 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.286642 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.288220 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.288663 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.288797 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.292877 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.292890 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.292906 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.292941 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.296051 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.296655 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.296662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.296664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.296783 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.297081 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.297084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.297085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.297420 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.297655 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.297817 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.297971 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":272,"threads_started":1}
I20260811 06:23:03.298844 8218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:03.390308 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8076
I20260811 06:23:03.412108 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8231
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[7801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:03.427197 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:03.413025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.7.4.131:37681)
0811 06:23:03.413218 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:03.413226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:03.413265 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:23:03.413467 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:03.413473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:03.413550 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:03.413677 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:03.414941 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.415800 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.417035 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:03.417437 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:03.417587 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:03.421770 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:03.421798 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:03.421812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:03.421849 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:03.424456 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.425128 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.425137 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.425140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.425280 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:03.425683 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:03.425690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:03.425697 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:03.426028 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:03.426330 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:03.426620 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 06:23:03.426856 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:23:03.427963 8362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:03.517185 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8231
I20260811 06:23:03.538491 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 7814
I20260811 06:23:03.539780 7884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:03.655531 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 7814
2026-08-11T06:23:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4556 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[8425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[8425](info): set up 2 sockets
Aug 11 06:23:03 dist-test-slave-07c2 krb5kdc[8425](info): commencing operation
krb5kdc: starting...
W20260811 06:23:04.801175 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:04 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:05.004894 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42597
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:42597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:05.335546 8438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:05.335870 8438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:05.336067 8438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:05.343478 8438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:05.343593 8438 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:23:05.343703 8438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:05.343791 8438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:05.343894 8438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:05.344007 8438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:05.354350 8438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:42597
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:05.355772 8438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:05.357235 8438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:05.369442 8444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:05.369635 8443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:05.370106 8446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:05.370182 8438 server_base.cc:1034] running on GCE node
I20260811 06:23:05.371028 8438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:05.372581 8438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:05.373800 8438 hybrid_clock.cc:648] HybridClock initialized: now 1786429385373727 us; error 47 us; skew 500 ppm
I20260811 06:23:05.374142 8438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:05 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:05.377332 8438 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:05.378844 8438 webserver.cc:460] Webserver started at http://127.7.4.190:44927/ using document root <none> and password file <none>
I20260811 06:23:05.379624 8438 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:05.379765 8438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:05.380141 8438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:05.383198 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "4a3c064dc719413089e8d6868ab517b6"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "e2bab20bcb411fec40a758aa8dd01c47"
server_key_iv: "e8702755f3bb88ef8b8969ba02a59f01"
server_key_version: "encryptionkey@0"
I20260811 06:23:05.384127 8438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "4a3c064dc719413089e8d6868ab517b6"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "e2bab20bcb411fec40a758aa8dd01c47"
server_key_iv: "e8702755f3bb88ef8b8969ba02a59f01"
server_key_version: "encryptionkey@0"
I20260811 06:23:05.389360 8438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:05.392792 8455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:05.394094 8438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:05.394304 8438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "4a3c064dc719413089e8d6868ab517b6"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "e2bab20bcb411fec40a758aa8dd01c47"
server_key_iv: "e8702755f3bb88ef8b8969ba02a59f01"
server_key_version: "encryptionkey@0"
I20260811 06:23:05.394523 8438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:05.408344 8438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:05.412441 8438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:05.412726 8438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:05.436470 8438 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:42597
I20260811 06:23:05.436488 8521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:42597 every 8 connection(s)
I20260811 06:23:05.437969 8438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:05.441758 8522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:05.444934 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8438
I20260811 06:23:05.445127 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:05.445799 7186 external_mini_cluster.cc:1428] Setting key c8909821e16b35c66a8d7280a7fa366d
I20260811 06:23:05.455127 8522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Bootstrap starting.
I20260811 06:23:05.458773 8522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:05.459861 8522 log.cc:824] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:23:05 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:05.463083 8522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: No bootstrap required, opened a new log
I20260811 06:23:05.470018 8525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:05.447903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.7.4.190:42597)
0811 06:23:05.448793 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:05.448808 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:05.448858 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:23:05.450839 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:05.450873 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:05.451419 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:05.451861 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:05.453225 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:05.455223 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:05.456514 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:05.457610 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:05.457806 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:05.462463 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:05.462470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:05.462485 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:05.462521 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:05.465713 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:05.466555 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:05.466562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:05.466568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:05.466741 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:05.467163 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:05.467167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:05.467169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:05.467788 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:05.468030 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:05.468487 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 06:23:05.468934 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":369,"threads_started":1}
I20260811 06:23:05.472385 8522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } }
I20260811 06:23:05.472767 8522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:05.472910 8522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a3c064dc719413089e8d6868ab517b6, State: Initialized, Role: FOLLOWER
I20260811 06:23:05.473467 8522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } }
I20260811 06:23:05.473681 8522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:05.473779 8522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:05.473897 8522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:05.476343 8522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } }
I20260811 06:23:05.476820 8522 leader_election.cc:302] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a3c064dc719413089e8d6868ab517b6; no voters:
I20260811 06:23:05.477319 8522 leader_election.cc:288] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:05.477468 8528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:05.478279 8528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [term 1 LEADER]: Becoming Leader. State: Replica: 4a3c064dc719413089e8d6868ab517b6, State: Running, Role: LEADER
I20260811 06:23:05.478857 8528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } }
I20260811 06:23:05.479669 8522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:05.482159 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a3c064dc719413089e8d6868ab517b6. Latest consensus state: current_term: 1 leader_uuid: "4a3c064dc719413089e8d6868ab517b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } } }
I20260811 06:23:05.482280 8529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a3c064dc719413089e8d6868ab517b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3c064dc719413089e8d6868ab517b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42597 } } }
I20260811 06:23:05.482600 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:05.482668 8529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:05.485893 8534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:05.493155 8534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:05.503831 8534 catalog_manager.cc:1347] Generated new cluster ID: 64848c89b34745a49f1abc9cefb6008c
I20260811 06:23:05.503984 8534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:05.523674 8534 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:05.525131 8534 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:05.539160 8534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Generated new TSK 0
I20260811 06:23:05.540025 8534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:05.600322 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42597
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:05.948057 8552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:05.948320 8552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:05.948488 8552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:05.955555 8552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:05.955646 8552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:05.955816 8552 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:05.955928 8552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:05.965684 8552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:42597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:05.966904 8552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:05.968251 8552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:05.980711 8559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:05.980803 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:05.980824 8561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:05.982021 8552 server_base.cc:1034] running on GCE node
I20260811 06:23:05.982591 8552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:05.983507 8552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:05.984755 8552 hybrid_clock.cc:648] HybridClock initialized: now 1786429385984696 us; error 35 us; skew 500 ppm
I20260811 06:23:05.985047 8552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:05 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:05.993745 8552 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:05.995447 8552 webserver.cc:460] Webserver started at http://127.7.4.129:38171/ using document root <none> and password file <none>
I20260811 06:23:05.996150 8552 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:05.996286 8552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:05.996620 8552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:05.999637 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "7a57a85ea9f202db6680e53fcb7cdbce"
server_key_iv: "c5779f895dad12146db0ed787516d319"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.000418 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "7a57a85ea9f202db6680e53fcb7cdbce"
server_key_iv: "c5779f895dad12146db0ed787516d319"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.005366 8552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:06.008862 8568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:06.010190 8552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:06.010370 8552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f"
format_stamp: "Formatted at 2026-08-11 06:23:05 on dist-test-slave-07c2"
server_key: "7a57a85ea9f202db6680e53fcb7cdbce"
server_key_iv: "c5779f895dad12146db0ed787516d319"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.010614 8552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:06.023193 8552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:06.027575 8552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:06.027863 8552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:06.030637 8552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:06.030773 8552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:06.030934 8552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:06.031004 8552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:06 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:06.055876 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.031588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.129:45811)
0811 06:23:06.032477 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:06.032529 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:06.034969 (+ 2440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:06.035876 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:06.035886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:06.036676 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:06.037516 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.037549 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.039161 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.039165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.041077 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.041092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.042372 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.042377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.042539 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.043342 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:06.043373 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:06.046316 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:06.050797 (+ 4481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.050808 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.050836 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.051444 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.051923 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.051931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.051937 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.052275 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.053052 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:06.053057 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:06.053672 (+ 615us) client_negotiation.cc:769] Sending connection context
0811 06:23:06.053972 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 06:23:06.054468 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":383,"threads_started":1}
I20260811 06:23:06.056356 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.031017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45811 (local address 127.7.4.190:42597)
0811 06:23:06.031532 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:06.031538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:06.032672 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 06:23:06.035329 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:06.035335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:06.035452 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:06.035587 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:06.037770 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.038900 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.041307 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.041899 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.041998 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.046548 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:06.046557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:06.046564 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:06.046616 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:06.050636 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.051610 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.051618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.051623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.051793 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.052433 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.052437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.052439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.052836 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:06.052987 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:06.055534 (+ 2547us) server_negotiation.cc:299] Negotiation successful
0811 06:23:06.055847 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260811 06:23:06.076145 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.061791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:36087 (local address 127.7.4.190:42597)
0811 06:23:06.062028 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:06.062036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:06.062607 (+ 571us) server_negotiation.cc:407] Connection header received
0811 06:23:06.062758 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:06.062766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:06.062861 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:06.062979 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:06.063933 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.065340 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.066940 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.067464 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.067565 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.070071 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:06.070076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:06.070080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:06.070108 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:06.073440 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.074145 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.074151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.074154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.074287 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.074640 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.074647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.074652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.074931 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:06.075092 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:06.075545 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:23:06.075768 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:23:06.076275 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.061790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.129:36087)
0811 06:23:06.062515 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:06.062531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:06.062645 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:06.063010 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:06.063013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:06.063490 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:06.063791 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.063800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.065487 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.065490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.066741 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.066752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.067906 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.067910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.068058 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.068801 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:06.068817 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:06.069877 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:06.073634 (+ 3757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.073640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.073644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.074021 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.074400 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.074406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.074409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.074536 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.075055 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:06.075060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:06.075242 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 06:23:06.075448 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 06:23:06.075790 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":430,"threads_started":1}
I20260811 06:23:06.087778 8552 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:42745
I20260811 06:23:06.087801 8700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:42745 every 8 connection(s)
I20260811 06:23:06.089226 8552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:06.091348 8701 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42597
I20260811 06:23:06.091543 8701 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:06.092114 8701 heartbeater.cc:511] Master 127.7.4.190:42597 requested a full tablet report, sending...
I20260811 06:23:06.093982 8474 ts_manager.cc:194] Registered new tserver with Master: c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129:42745)
I20260811 06:23:06.094278 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8552
I20260811 06:23:06.094475 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:06.095323 7186 external_mini_cluster.cc:1428] Setting key 507d827483d828f14caacf15e156f1e4
I20260811 06:23:06.096088 8474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:45811
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:06.140033 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42597
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:06.462970 8706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:06.463318 8706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:06.463454 8706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:06.470604 8706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:06.470700 8706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:06.470794 8706 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:06.470875 8706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:06.480527 8706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:42597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:06.481698 8706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:06.482946 8706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:06.495846 8711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:06.495812 8712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:06.495885 8715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:06.496680 8706 server_base.cc:1034] running on GCE node
I20260811 06:23:06.497277 8706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:06.498243 8706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:06.499469 8706 hybrid_clock.cc:648] HybridClock initialized: now 1786429386499410 us; error 34 us; skew 500 ppm
I20260811 06:23:06.499760 8706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:06 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:06.502918 8706 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:06.504451 8706 webserver.cc:460] Webserver started at http://127.7.4.130:43193/ using document root <none> and password file <none>
I20260811 06:23:06.505084 8706 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:06.505173 8706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:06.505445 8706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:06.508551 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "2455268dce114057a0cef8178b9df052"
format_stamp: "Formatted at 2026-08-11 06:23:06 on dist-test-slave-07c2"
server_key: "ea178446b54f14940a36d68d9d2bc71a"
server_key_iv: "1aecda47fffc0131078587cac413a1d3"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.509308 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "2455268dce114057a0cef8178b9df052"
format_stamp: "Formatted at 2026-08-11 06:23:06 on dist-test-slave-07c2"
server_key: "ea178446b54f14940a36d68d9d2bc71a"
server_key_iv: "1aecda47fffc0131078587cac413a1d3"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.514147 8706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:23:06.517552 8724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:06.518847 8706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:23:06.518978 8706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "2455268dce114057a0cef8178b9df052"
format_stamp: "Formatted at 2026-08-11 06:23:06 on dist-test-slave-07c2"
server_key: "ea178446b54f14940a36d68d9d2bc71a"
server_key_iv: "1aecda47fffc0131078587cac413a1d3"
server_key_version: "encryptionkey@0"
I20260811 06:23:06.519176 8706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:06.535866 8706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:06.540261 8706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:06.540479 8706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:06.543116 8706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:06.543231 8706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:06.543416 8706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:06.543486 8706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:06 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429386, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:06.566421 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.543490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:49121 (local address 127.7.4.190:42597)
0811 06:23:06.543700 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:06.543706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:06.545097 (+ 1391us) server_negotiation.cc:407] Connection header received
0811 06:23:06.547457 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:06.547463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:06.547565 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:06.547717 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:06.550193 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.551259 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.553959 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.554523 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.554661 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.559529 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:06.559533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:06.559537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:06.559571 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:06.562871 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.563952 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.563961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.563965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.564093 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.564548 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.564553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.564556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.564845 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:06.565021 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:06.565729 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 06:23:06.565952 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:23:06.567576 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.543973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.130:49121)
0811 06:23:06.544903 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:06.544952 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:06.547086 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:06.548122 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:06.548135 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:06.548961 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:06.549922 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.549951 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.551485 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.551494 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.553792 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.553813 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.554903 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.554914 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.555297 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.556132 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:06.556170 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:06.559271 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:06.563071 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.563079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.563099 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.563823 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.564211 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.564216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.564219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.564425 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.564980 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:06.564989 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:06.565351 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 06:23:06.565609 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 06:23:06.566224 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":399,"threads_started":1}
I20260811 06:23:06.589515 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.573622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:60685 (local address 127.7.4.190:42597)
0811 06:23:06.573797 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:06.573801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:06.574549 (+ 748us) server_negotiation.cc:407] Connection header received
0811 06:23:06.574761 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:06.574769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:06.574861 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:06.575005 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:06.576420 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.577730 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.579620 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.580024 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.580175 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.582831 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:06.582836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:06.582839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:06.582867 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:06.586000 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.587102 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.587109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.587113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.587279 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:06.587849 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:06.587857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:06.587863 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:06.588276 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:06.588450 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:06.588927 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 06:23:06.589155 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:23:06.589681 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:06.573665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.130:60685)
0811 06:23:06.574379 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:06.574416 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:06.574580 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:06.575037 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:06.575047 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:06.575885 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:06.576245 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.576254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.577916 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.577923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.579454 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:06.579467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:06.580329 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:06.580338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:06.580633 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:06.581334 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:06.581364 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:06.582626 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:06.586206 (+ 3580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.586214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.586217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.586918 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.587454 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:06.587461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:06.587468 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:06.587700 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:06.588421 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:06.588429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:06.588678 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 06:23:06.588833 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 06:23:06.589191 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":418,"threads_started":1}
I20260811 06:23:06.600725 8706 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:36263
I20260811 06:23:06.600747 8855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:36263 every 8 connection(s)
I20260811 06:23:06.601946 8706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:06.603911 8856 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42597
I20260811 06:23:06.604167 8856 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:06.604823 8856 heartbeater.cc:511] Master 127.7.4.190:42597 requested a full tablet report, sending...
I20260811 06:23:06.606124 8472 ts_manager.cc:194] Registered new tserver with Master: 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263)
I20260811 06:23:06.606978 8472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:49121
I20260811 06:23:06.609162 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8706
I20260811 06:23:06.609328 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:06.609783 7186 external_mini_cluster.cc:1428] Setting key c03dae6c9f653ebe201cfca7b701ed30
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:06.658324 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42597
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:06.982915 8861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:06.983192 8861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:06.983336 8861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:06.990716 8861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:06.990823 8861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:06.990906 8861 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:06.990994 8861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:07.000944 8861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:42597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:07.002151 8861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:07.003477 8861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:07.016948 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:07.017030 8867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:07.016970 8866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:07.017958 8861 server_base.cc:1034] running on GCE node
I20260811 06:23:07.018612 8861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:07.019531 8861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:07.020756 8861 hybrid_clock.cc:648] HybridClock initialized: now 1786429387020686 us; error 39 us; skew 500 ppm
I20260811 06:23:07.021055 8861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:07 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:07.024459 8861 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:07.025930 8861 webserver.cc:460] Webserver started at http://127.7.4.131:42005/ using document root <none> and password file <none>
I20260811 06:23:07.026608 8861 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:07.026734 8861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:07.027073 8861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:07.030175 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "62b9f74de33b4d408c6494e3e20638c4"
format_stamp: "Formatted at 2026-08-11 06:23:07 on dist-test-slave-07c2"
server_key: "2a4dd1fca5d0c16f2497ec068b5ddc7f"
server_key_iv: "13d402b5deb422167b72bd179061bff7"
server_key_version: "encryptionkey@0"
I20260811 06:23:07.031000 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "62b9f74de33b4d408c6494e3e20638c4"
format_stamp: "Formatted at 2026-08-11 06:23:07 on dist-test-slave-07c2"
server_key: "2a4dd1fca5d0c16f2497ec068b5ddc7f"
server_key_iv: "13d402b5deb422167b72bd179061bff7"
server_key_version: "encryptionkey@0"
I20260811 06:23:07.036146 8861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:23:07.039480 8876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:07.040792 8861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:23:07.040964 8861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "62b9f74de33b4d408c6494e3e20638c4"
format_stamp: "Formatted at 2026-08-11 06:23:07 on dist-test-slave-07c2"
server_key: "2a4dd1fca5d0c16f2497ec068b5ddc7f"
server_key_iv: "13d402b5deb422167b72bd179061bff7"
server_key_version: "encryptionkey@0"
I20260811 06:23:07.041177 8861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:07.061287 8861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:07.065177 8861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:07.065467 8861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:07.068261 8861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:07.068374 8861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:07.068516 8861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:07.068617 8861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:07 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:07.092329 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.068600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:52559 (local address 127.7.4.190:42597)
0811 06:23:07.068832 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.068842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.070506 (+ 1664us) server_negotiation.cc:407] Connection header received
0811 06:23:07.073724 (+ 3218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.073730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.073820 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.073926 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:07.076299 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.077397 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.080044 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.080461 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.080622 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.085309 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:07.085314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:07.085317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:07.085351 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:07.088596 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.089525 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.089537 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.089540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.089736 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.090380 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.090385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.090388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.090752 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:07.090928 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.091725 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.091964 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:23:07.093381 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.069256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.131:52559)
0811 06:23:07.070307 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.070363 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.073317 (+ 2954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.074306 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.074320 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.075147 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:07.076026 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.076059 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.077560 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.077575 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.079902 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.079914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.080777 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.080787 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.081053 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.081956 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:07.081999 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:07.085058 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:07.088736 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.088745 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.088766 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.089401 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.089886 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.089891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.089896 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.090246 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.090887 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:07.090897 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:07.091251 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.091592 (+ 341us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.092123 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":391,"threads_started":1}
I20260811 06:23:07.098932 8701 heartbeater.cc:503] Master 127.7.4.190:42597 was elected leader, sending a full tablet report...
I20260811 06:23:07.118296 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.100312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:52881 (local address 127.7.4.190:42597)
0811 06:23:07.100500 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.100508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.101193 (+ 685us) server_negotiation.cc:407] Connection header received
0811 06:23:07.101375 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.101383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.101501 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.101627 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:07.102517 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.103917 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.106771 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.107621 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.107742 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.111475 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:07.111483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:07.111487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:07.111528 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:07.115044 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.116236 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.116241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.116243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.116373 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.116823 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.116826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.116828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.117107 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:07.117225 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.117857 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.118033 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:23:07.118971 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.100335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.7.4.131:52881)
0811 06:23:07.101058 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.101080 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.101236 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.101637 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.101640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.102081 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:07.102378 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.102387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.104127 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.104147 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.106542 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.106565 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.108127 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.108145 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.108468 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.109486 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:07.109523 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:07.111109 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:07.115211 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.115219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.115226 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.116093 (+ 867us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.116504 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.116512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.116517 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.116721 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.117272 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:07.117280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:07.117561 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.117747 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.118265 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":399,"threads_started":1}
I20260811 06:23:07.132354 8861 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:33687
I20260811 06:23:07.132387 9005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:33687 every 8 connection(s)
I20260811 06:23:07.133626 8861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:07.135950 9006 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42597
I20260811 06:23:07.136197 9006 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:07.136861 9006 heartbeater.cc:511] Master 127.7.4.190:42597 requested a full tablet report, sending...
I20260811 06:23:07.137413 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 8861
I20260811 06:23:07.137573 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:07.138144 7186 external_mini_cluster.cc:1428] Setting key 0067fbd68ffaeb450ebdc62ca177f655
I20260811 06:23:07.138275 8472 ts_manager.cc:194] Registered new tserver with Master: 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687)
I20260811 06:23:07.139443 8472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:52559
I20260811 06:23:07.141150 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:23:07 dist-test-slave-07c2 krb5kdc[8425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:23:07 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:07.194307 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.180136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.7.4.190:42597)
0811 06:23:07.180385 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.180392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.180705 (+ 313us) server_negotiation.cc:407] Connection header received
0811 06:23:07.180924 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.180929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.181014 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.181150 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:07.182533 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.183358 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.184646 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.185036 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.185169 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.188983 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:07.188988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:07.188991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:07.189025 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:07.191655 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.192305 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.192317 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.192319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.192447 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.192836 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.192842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.192845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.193121 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:07.193394 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.193696 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.193930 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:23:07.198464 8472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:07.205340 8472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:07.232232 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.218676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42745 (local address 127.0.0.1:37136)
0811 06:23:07.221659 (+ 2983us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.221681 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.221815 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.222826 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.222837 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.222881 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:07.223458 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.223475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.225826 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.225831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.228897 (+ 3066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.228919 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.231109 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.231118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.231437 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.231511 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.231676 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.231888 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2832,"thread_start_us":11652,"threads_started":1}
I20260811 06:23:07.232405 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.219605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.7.4.129:42745)
0811 06:23:07.220105 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.220111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.221948 (+ 1837us) server_negotiation.cc:407] Connection header received
0811 06:23:07.222066 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.222074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.222351 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.222554 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:07.223740 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.225673 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.229135 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.230841 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.231014 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.231337 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.231768 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.231996 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":249,"threads_started":1}
I20260811 06:23:07.236721 8624 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddd004ff47442caa90ef12bc6017131 (DEFAULT_TABLE table=test-table [id=67ed1369aa5a4275b52f883ac9d18426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.238358 8624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddd004ff47442caa90ef12bc6017131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.240306 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.219109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.7.4.130:36263)
0811 06:23:07.232093 (+ 12984us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.232101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.232130 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:23:07.232240 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.232249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.233027 (+ 778us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.233268 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:07.233822 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.235255 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.237711 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.239089 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.239222 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.239446 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.239820 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.240001 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12797,"thread_start_us":13559,"threads_started":1}
I20260811 06:23:07.240784 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.218551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36263 (local address 127.0.0.1:39324)
0811 06:23:07.219265 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.219327 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.219509 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.233224 (+ 13715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.233242 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.233268 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:07.233569 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.233604 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.235449 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.235454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.237467 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.237482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.239336 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.239346 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.239583 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.239629 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.239745 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.239995 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":80,"thread_start_us":1524,"threads_started":1}
I20260811 06:23:07.243502 9022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.221004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33687 (local address 127.0.0.1:60768)
0811 06:23:07.230048 (+ 9044us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.230073 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.230224 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.237303 (+ 7079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.237308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.237339 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:07.237800 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.237813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.239390 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.239394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.241393 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.241414 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.242675 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.242682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.242872 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.242911 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.243021 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.243149 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8863,"thread_start_us":327,"threads_started":1}
I20260811 06:23:07.243554 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.221405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.7.4.131:33687)
0811 06:23:07.236854 (+ 15449us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.236860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.236886 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:23:07.236959 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.236964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.237116 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.237316 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:07.238020 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.239201 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.241558 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.242473 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.242616 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.242876 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.243066 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.243215 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15301,"thread_start_us":16176,"threads_started":1}
I20260811 06:23:07.244630 8780 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddd004ff47442caa90ef12bc6017131 (DEFAULT_TABLE table=test-table [id=67ed1369aa5a4275b52f883ac9d18426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.246138 8780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddd004ff47442caa90ef12bc6017131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.246387 8937 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddd004ff47442caa90ef12bc6017131 (DEFAULT_TABLE table=test-table [id=67ed1369aa5a4275b52f883ac9d18426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.247524 8937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddd004ff47442caa90ef12bc6017131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.255172 9027 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Bootstrap starting.
I20260811 06:23:07.259291 9027 tablet_bootstrap.cc:654] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.259508 9028 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Bootstrap starting.
I20260811 06:23:07.260721 9027 log.cc:824] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:07.261107 9029 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Bootstrap starting.
I20260811 06:23:07.262966 9028 tablet_bootstrap.cc:654] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.263950 9028 log.cc:824] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:07.265347 9029 tablet_bootstrap.cc:654] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.266532 9028 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: No bootstrap required, opened a new log
I20260811 06:23:07.266875 9028 ts_tablet_manager.cc:1397] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 06:23:07.266954 9029 log.cc:824] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:07.270291 9029 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: No bootstrap required, opened a new log
I20260811 06:23:07.270407 9027 tablet_bootstrap.cc:492] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: No bootstrap required, opened a new log
I20260811 06:23:07.270597 9029 ts_tablet_manager.cc:1397] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 06:23:07.270677 9027 ts_tablet_manager.cc:1397] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260811 06:23:07.278082 9029 raft_consensus.cc:348] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.278091 9028 raft_consensus.cc:348] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.278453 9029 raft_consensus.cc:374] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.278540 9028 raft_consensus.cc:374] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.278549 9029 raft_consensus.cc:729] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2455268dce114057a0cef8178b9df052, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.278642 9028 raft_consensus.cc:729] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b9f74de33b4d408c6494e3e20638c4, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.279239 9028 consensus_queue.cc:260] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.279323 9029 consensus_queue.cc:260] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.281208 9028 ts_tablet_manager.cc:1428] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 06:23:07.281852 9006 heartbeater.cc:503] Master 127.7.4.190:42597 was elected leader, sending a full tablet report...
I20260811 06:23:07.282022 9027 raft_consensus.cc:348] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.282364 9027 raft_consensus.cc:374] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.282500 9027 raft_consensus.cc:729] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e64fc5e9e14d89b5ddc5475e3e592f, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.282996 9027 consensus_queue.cc:260] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.287585 9027 ts_tablet_manager.cc:1428] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent starting tablet: real 0.017s user 0.012s sys 0.006s
I20260811 06:23:07.288407 8856 heartbeater.cc:503] Master 127.7.4.190:42597 was elected leader, sending a full tablet report...
I20260811 06:23:07.289481 9029 ts_tablet_manager.cc:1428] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Time spent starting tablet: real 0.019s user 0.012s sys 0.004s
W20260811 06:23:07.341974 8703 tablet.cc:2369] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:07.353971 8857 tablet.cc:2369] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:07.385289 9007 tablet.cc:2369] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:07.610229 9036 raft_consensus.cc:482] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:07.610553 9036 raft_consensus.cc:504] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.612351 9036 leader_election.cc:288] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263), 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687)
I20260811 06:23:07.619220 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.612802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:50267 (local address 127.7.4.130:36263)
0811 06:23:07.612986 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.612990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.613549 (+ 559us) server_negotiation.cc:407] Connection header received
0811 06:23:07.613930 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.613935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.614011 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.614092 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:07.614805 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.615648 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.617521 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.618284 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.618430 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.618538 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.618812 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.618960 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:07.619572 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.613581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45355 (local address 127.7.4.131:33687)
0811 06:23:07.613790 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.613798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.614221 (+ 423us) server_negotiation.cc:407] Connection header received
0811 06:23:07.614419 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.614445 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.614529 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.614627 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:07.615274 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.616026 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.617829 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.618573 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.618725 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.618838 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.619171 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.619314 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:07.619611 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.612637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36263 (local address 127.7.4.129:50267)
0811 06:23:07.613435 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.613457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.613712 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.614313 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.614322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.614345 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:07.614660 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.614668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.615772 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.615779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.617364 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.617376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.618498 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.618504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.618652 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.618676 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.618786 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.618963 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":73,"thread_start_us":321,"threads_started":1}
I20260811 06:23:07.620050 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.613438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33687 (local address 127.7.4.129:45355)
0811 06:23:07.614100 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.614114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.614284 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.614661 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.614665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.614693 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:07.615065 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.615096 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.616183 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.616188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.617677 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.617689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.618818 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.618825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.619018 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.619040 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.619140 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.619251 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":35,"thread_start_us":416,"threads_started":1}
I20260811 06:23:07.620738 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddd004ff47442caa90ef12bc6017131" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052" is_pre_election: true
I20260811 06:23:07.620854 8958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddd004ff47442caa90ef12bc6017131" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b9f74de33b4d408c6494e3e20638c4" is_pre_election: true
I20260811 06:23:07.621281 8958 raft_consensus.cc:2454] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 0.
I20260811 06:23:07.621253 8805 raft_consensus.cc:2454] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 0.
I20260811 06:23:07.621912 8571 leader_election.cc:302] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62b9f74de33b4d408c6494e3e20638c4, c2e64fc5e9e14d89b5ddc5475e3e592f; no voters:
I20260811 06:23:07.622290 9036 raft_consensus.cc:2781] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:07.622463 9036 raft_consensus.cc:482] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:07.622584 9036 raft_consensus.cc:3037] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.625311 9036 raft_consensus.cc:504] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.625874 9036 leader_election.cc:288] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 election: Requested vote from peers 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263), 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687)
I20260811 06:23:07.626456 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddd004ff47442caa90ef12bc6017131" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052"
I20260811 06:23:07.626506 8958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddd004ff47442caa90ef12bc6017131" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b9f74de33b4d408c6494e3e20638c4"
I20260811 06:23:07.626693 8805 raft_consensus.cc:3037] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.626688 8958 raft_consensus.cc:3037] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.630023 8805 raft_consensus.cc:2454] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 1.
I20260811 06:23:07.630024 8958 raft_consensus.cc:2454] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 1.
I20260811 06:23:07.630410 8571 leader_election.cc:302] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2455268dce114057a0cef8178b9df052, c2e64fc5e9e14d89b5ddc5475e3e592f; no voters:
I20260811 06:23:07.630674 9036 raft_consensus.cc:2781] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:07.631102 9036 raft_consensus.cc:686] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 LEADER]: Becoming Leader. State: Replica: c2e64fc5e9e14d89b5ddc5475e3e592f, State: Running, Role: LEADER
I20260811 06:23:07.631631 9036 consensus_queue.cc:237] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.636162 8472 catalog_manager.cc:5495] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f reported cstate change: term changed from 0 to 1, leader changed from <none> to c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129). New cstate: current_term: 1 leader_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } health_report { overall_health: UNKNOWN } } }
I20260811 06:23:07.694994 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.688854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.7.4.129:42745)
0811 06:23:07.689109 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.689116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.689204 (+ 88us) server_negotiation.cc:407] Connection header received
0811 06:23:07.689658 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.689663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.689777 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.689981 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:07.690369 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.691092 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.692557 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.693026 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.693169 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.693753 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:07.694313 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:07.694477 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.694577 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.694671 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 06:23:07.704843 8805 raft_consensus.cc:1261] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Refusing update from remote peer c2e64fc5e9e14d89b5ddc5475e3e592f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:07.705588 8958 raft_consensus.cc:1261] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Refusing update from remote peer c2e64fc5e9e14d89b5ddc5475e3e592f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:07.706049 9040 consensus_queue.cc:1035] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:07.706543 9036 consensus_queue.cc:1035] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Connected to new peer: Peer: permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:07.720425 9045 mvcc.cc:205] Tried to move back new op lower bound from 7317214772026052608 to 7317214771742457856. Current Snapshot: MvccSnapshot[applied={T|T < 7317214772026052608}]
I20260811 06:23:07.744156 8472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:07.744606 8472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:07.748443 8472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Sending DeleteTablet for 3 replicas of tablet 5ddd004ff47442caa90ef12bc6017131
I20260811 06:23:07.749997 8624 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddd004ff47442caa90ef12bc6017131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:07 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:37136
I20260811 06:23:07.750003 8780 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddd004ff47442caa90ef12bc6017131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:07 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:39324
I20260811 06:23:07.750327 8937 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddd004ff47442caa90ef12bc6017131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:07 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60768
I20260811 06:23:07.753299 8472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:07.754141 8472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:07.764257 9056 tablet_replica.cc:330] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: stopping tablet replica
I20260811 06:23:07.764981 9056 raft_consensus.cc:2229] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:07.765574 9056 raft_consensus.cc:2258] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:07.766085 8621 tablet_service.cc:1467] Processing CreateTablet for tablet 74de356281a9416cb11bc144ec6afd16 (DEFAULT_TABLE table=test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.766973 8621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74de356281a9416cb11bc144ec6afd16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.766868 8779 tablet_service.cc:1467] Processing CreateTablet for tablet 74de356281a9416cb11bc144ec6afd16 (DEFAULT_TABLE table=test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.766965 8935 tablet_service.cc:1467] Processing CreateTablet for tablet 74de356281a9416cb11bc144ec6afd16 (DEFAULT_TABLE table=test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:07.767411 8935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74de356281a9416cb11bc144ec6afd16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.767484 8779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74de356281a9416cb11bc144ec6afd16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.775068 9028 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Bootstrap starting.
I20260811 06:23:07.775084 9029 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Bootstrap starting.
I20260811 06:23:07.775084 9027 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Bootstrap starting.
I20260811 06:23:07.776502 9054 tablet_replica.cc:330] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: stopping tablet replica
I20260811 06:23:07.776885 9027 tablet_bootstrap.cc:654] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.776887 9029 tablet_bootstrap.cc:654] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.777086 9028 tablet_bootstrap.cc:654] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.777426 9054 raft_consensus.cc:2229] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:07.778203 9054 raft_consensus.cc:2258] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:07.780504 9054 ts_tablet_manager.cc:1905] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:07.787165 9028 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: No bootstrap required, opened a new log
I20260811 06:23:07.787420 9028 ts_tablet_manager.cc:1397] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260811 06:23:07.788607 9028 raft_consensus.cc:348] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.789016 9028 raft_consensus.cc:374] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.789148 9028 raft_consensus.cc:729] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b9f74de33b4d408c6494e3e20638c4, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.789486 9028 consensus_queue.cc:260] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.789575 9054 ts_tablet_manager.cc:1918] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:07.789899 9054 log.cc:1196] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/5ddd004ff47442caa90ef12bc6017131
I20260811 06:23:07.790251 9029 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: No bootstrap required, opened a new log
I20260811 06:23:07.790401 9029 ts_tablet_manager.cc:1397] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260811 06:23:07.790684 9028 ts_tablet_manager.cc:1428] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:23:07.791049 9054 ts_tablet_manager.cc:1939] T 5ddd004ff47442caa90ef12bc6017131 P 2455268dce114057a0cef8178b9df052: Deleting consensus metadata
I20260811 06:23:07.791430 9029 raft_consensus.cc:348] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.791736 9029 raft_consensus.cc:374] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.791863 9029 raft_consensus.cc:729] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2455268dce114057a0cef8178b9df052, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.792243 9029 consensus_queue.cc:260] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.793628 8458 catalog_manager.cc:4841] TS 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263): tablet 5ddd004ff47442caa90ef12bc6017131 (table test-table [id=67ed1369aa5a4275b52f883ac9d18426]) successfully deleted
I20260811 06:23:07.793926 9029 ts_tablet_manager.cc:1428] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:07.795755 9027 tablet_bootstrap.cc:492] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: No bootstrap required, opened a new log
I20260811 06:23:07.795923 9027 ts_tablet_manager.cc:1397] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.009s
I20260811 06:23:07.796892 9027 raft_consensus.cc:348] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.797156 9027 raft_consensus.cc:374] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.797374 9027 raft_consensus.cc:729] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e64fc5e9e14d89b5ddc5475e3e592f, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.797699 9027 consensus_queue.cc:260] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.798513 9056 ts_tablet_manager.cc:1905] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:07.799357 9027 ts_tablet_manager.cc:1428] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:07.800021 9055 tablet_replica.cc:330] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: stopping tablet replica
I20260811 06:23:07.800474 9055 raft_consensus.cc:2229] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:07.802649 9055 raft_consensus.cc:2258] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:07.804481 9055 ts_tablet_manager.cc:1905] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:07.806331 9056 ts_tablet_manager.cc:1918] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:07.806567 9056 log.cc:1196] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/5ddd004ff47442caa90ef12bc6017131
I20260811 06:23:07.807199 9056 ts_tablet_manager.cc:1939] T 5ddd004ff47442caa90ef12bc6017131 P 62b9f74de33b4d408c6494e3e20638c4: Deleting consensus metadata
I20260811 06:23:07.809177 8458 catalog_manager.cc:4841] TS 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687): tablet 5ddd004ff47442caa90ef12bc6017131 (table test-table [id=67ed1369aa5a4275b52f883ac9d18426]) successfully deleted
I20260811 06:23:07.810771 9055 ts_tablet_manager.cc:1918] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:07.810966 9055 log.cc:1196] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/5ddd004ff47442caa90ef12bc6017131
I20260811 06:23:07.811546 9055 ts_tablet_manager.cc:1939] T 5ddd004ff47442caa90ef12bc6017131 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting consensus metadata
I20260811 06:23:07.813411 8456 catalog_manager.cc:4841] TS c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129:42745): tablet 5ddd004ff47442caa90ef12bc6017131 (table test-table [id=67ed1369aa5a4275b52f883ac9d18426]) successfully deleted
I20260811 06:23:07.849249 9040 raft_consensus.cc:482] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:07.849519 9040 raft_consensus.cc:504] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.850301 9040 leader_election.cc:288] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263), 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687)
I20260811 06:23:07.850970 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74de356281a9416cb11bc144ec6afd16" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052" is_pre_election: true
I20260811 06:23:07.851056 8958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74de356281a9416cb11bc144ec6afd16" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b9f74de33b4d408c6494e3e20638c4" is_pre_election: true
I20260811 06:23:07.851259 8805 raft_consensus.cc:2454] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 0.
I20260811 06:23:07.851377 8958 raft_consensus.cc:2454] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 0.
I20260811 06:23:07.851796 8571 leader_election.cc:302] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62b9f74de33b4d408c6494e3e20638c4, c2e64fc5e9e14d89b5ddc5475e3e592f; no voters:
I20260811 06:23:07.852160 9040 raft_consensus.cc:2781] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:07.852300 9040 raft_consensus.cc:482] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:07.852535 9040 raft_consensus.cc:3037] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.855615 9040 raft_consensus.cc:504] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.856230 9040 leader_election.cc:288] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 election: Requested vote from peers 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263), 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687)
I20260811 06:23:07.856737 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74de356281a9416cb11bc144ec6afd16" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052"
I20260811 06:23:07.856798 8958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74de356281a9416cb11bc144ec6afd16" candidate_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b9f74de33b4d408c6494e3e20638c4"
I20260811 06:23:07.856945 8805 raft_consensus.cc:3037] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.857050 8958 raft_consensus.cc:3037] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:07.859553 8805 raft_consensus.cc:2454] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 1.
I20260811 06:23:07.860046 8571 leader_election.cc:302] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2455268dce114057a0cef8178b9df052, c2e64fc5e9e14d89b5ddc5475e3e592f; no voters:
I20260811 06:23:07.860287 8958 raft_consensus.cc:2454] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e64fc5e9e14d89b5ddc5475e3e592f in term 1.
I20260811 06:23:07.860330 9040 raft_consensus.cc:2781] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:07.860881 9040 raft_consensus.cc:686] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 LEADER]: Becoming Leader. State: Replica: c2e64fc5e9e14d89b5ddc5475e3e592f, State: Running, Role: LEADER
I20260811 06:23:07.861152 9040 consensus_queue.cc:237] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.864405 8472 catalog_manager.cc:5495] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f reported cstate change: term changed from 0 to 1, leader changed from <none> to c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129). New cstate: current_term: 1 leader_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:23:07 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429385, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:07.915326 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.898051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42597 (local address 127.0.0.1:60420)
0811 06:23:07.899111 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:07.899138 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:07.899358 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:07.900156 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:07.900160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:07.900907 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:07.901137 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.901146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.902742 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.902746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.903965 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:07.903977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.904947 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.904951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:07.905096 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.905863 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:07.905880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:07.908339 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:07.912596 (+ 4257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.912608 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.912613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.913058 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.913522 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:07.913525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:07.913528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:07.913685 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:07.914322 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:07.914328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:07.914488 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 06:23:07.914647 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 06:23:07.914851 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"spinlock_wait_cycles":4352,"thread_start_us":471,"threads_started":1}
I20260811 06:23:07.915715 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:07.898076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.7.4.190:42597)
0811 06:23:07.899806 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:07.899812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:07.899836 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:23:07.899893 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:07.899898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:07.899972 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:07.900089 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:07.901303 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.902564 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.904100 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:07.904755 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:07.904885 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:07.908571 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:07.908578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:07.908583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:07.908628 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:07.912418 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.913197 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.913203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.913207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.913382 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:07.913843 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:07.913848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:07.913851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:07.914196 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:07.914349 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:07.914790 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 06:23:07.915018 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"thread_start_us":1438,"threads_started":1}
I20260811 06:23:07.921288 8472 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60420:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:23:07.922678 8472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:07.930487 8624 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb7415e3ad48a287ebf65ff1f20985 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c2ba9b917d4b51ae8966a59a946605]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:07.930625 8779 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb7415e3ad48a287ebf65ff1f20985 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c2ba9b917d4b51ae8966a59a946605]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:07.931178 8779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb7415e3ad48a287ebf65ff1f20985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.931193 8624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb7415e3ad48a287ebf65ff1f20985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.931723 8937 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb7415e3ad48a287ebf65ff1f20985 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9c2ba9b917d4b51ae8966a59a946605]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:07.932232 8937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb7415e3ad48a287ebf65ff1f20985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:07.937929 9028 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: Bootstrap starting.
I20260811 06:23:07.939199 9027 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: Bootstrap starting.
I20260811 06:23:07.939842 9028 tablet_bootstrap.cc:654] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.941085 9027 tablet_bootstrap.cc:654] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.947939 9029 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: Bootstrap starting.
I20260811 06:23:07.949726 9029 tablet_bootstrap.cc:654] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:07.949911 9028 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: No bootstrap required, opened a new log
I20260811 06:23:07.950115 9028 ts_tablet_manager.cc:1397] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 06:23:07.950925 9028 raft_consensus.cc:348] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.951234 9028 raft_consensus.cc:374] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.951354 9028 raft_consensus.cc:729] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b9f74de33b4d408c6494e3e20638c4, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.951711 9028 consensus_queue.cc:260] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.952330 9029 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: No bootstrap required, opened a new log
I20260811 06:23:07.952250 9034 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:07.952504 9029 ts_tablet_manager.cc:1397] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260811 06:23:07.952613 9034 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:07.953261 9029 raft_consensus.cc:348] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.953521 9029 raft_consensus.cc:374] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.953604 9029 raft_consensus.cc:729] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2455268dce114057a0cef8178b9df052, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.953675 9028 ts_tablet_manager.cc:1428] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:07.953907 9029 consensus_queue.cc:260] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.954563 9029 ts_tablet_manager.cc:1428] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 06:23:07.955078 9035 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:07.955987 9035 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:07.956777 9027 tablet_bootstrap.cc:492] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: No bootstrap required, opened a new log
I20260811 06:23:07.956981 9027 ts_tablet_manager.cc:1397] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.007s
I20260811 06:23:07.958521 9027 raft_consensus.cc:348] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.958815 9027 raft_consensus.cc:374] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:07.958913 9027 raft_consensus.cc:729] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e64fc5e9e14d89b5ddc5475e3e592f, State: Initialized, Role: FOLLOWER
I20260811 06:23:07.959172 9027 consensus_queue.cc:260] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:07.959951 9027 ts_tablet_manager.cc:1428] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:07.959816 9040 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:07.960114 9040 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:08.108376 9034 raft_consensus.cc:482] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:08.108650 9034 raft_consensus.cc:504] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:08.110299 9034 leader_election.cc:288] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129:42745), 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263)
I20260811 06:23:08.129781 9071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.110503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42745 (local address 127.7.4.131:44541)
0811 06:23:08.121419 (+ 10916us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:08.121448 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:08.121948 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:08.122658 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:08.122664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:08.122700 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:08.122964 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.122976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.124681 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.124688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.127011 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.127027 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.128816 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.128823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.129030 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.129068 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:23:08.129183 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:23:08.129333 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10742,"mutex_wait_us":1,"thread_start_us":11094,"threads_started":1}
I20260811 06:23:08.129871 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.111792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:44541 (local address 127.7.4.129:42745)
0811 06:23:08.112030 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.112039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.121569 (+ 9530us) server_negotiation.cc:407] Connection header received
0811 06:23:08.122170 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.122180 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.122276 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.122358 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:08.123162 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.124463 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.127207 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.128608 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.128771 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.128891 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.129231 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.129391 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:23:08.130537 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.111183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:46187 (local address 127.7.4.130:36263)
0811 06:23:08.111442 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.111452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.123417 (+ 11965us) server_negotiation.cc:407] Connection header received
0811 06:23:08.123712 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.123737 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.123831 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.123894 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:08.124746 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.125676 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.127801 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.129258 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.129375 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.129465 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.130078 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.130232 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:23:08.131127 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.110771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36263 (local address 127.7.4.131:46187)
0811 06:23:08.123173 (+ 12402us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:08.123327 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:08.123538 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:08.124121 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:08.124125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:08.124150 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:08.124527 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.124543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.125856 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.125860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.127556 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.127573 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.129657 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.129664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.129887 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.129929 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 06:23:08.130670 (+ 741us) client_negotiation.cc:240] Negotiation successful
0811 06:23:08.130826 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12289,"thread_start_us":12098,"threads_started":1}
I20260811 06:23:08.131349 8652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb7415e3ad48a287ebf65ff1f20985" candidate_uuid: "62b9f74de33b4d408c6494e3e20638c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" is_pre_election: true
I20260811 06:23:08.131824 8652 raft_consensus.cc:2454] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62b9f74de33b4d408c6494e3e20638c4 in term 0.
I20260811 06:23:08.131875 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb7415e3ad48a287ebf65ff1f20985" candidate_uuid: "62b9f74de33b4d408c6494e3e20638c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052" is_pre_election: true
I20260811 06:23:08.132133 8805 raft_consensus.cc:2454] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62b9f74de33b4d408c6494e3e20638c4 in term 0.
I20260811 06:23:08.132648 8877 leader_election.cc:302] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2455268dce114057a0cef8178b9df052, 62b9f74de33b4d408c6494e3e20638c4, c2e64fc5e9e14d89b5ddc5475e3e592f; no voters:
I20260811 06:23:08.133180 9034 raft_consensus.cc:2781] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:08.133323 9034 raft_consensus.cc:482] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:08.133431 9034 raft_consensus.cc:3037] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:08.136032 9034 raft_consensus.cc:504] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:08.136613 9034 leader_election.cc:288] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [CANDIDATE]: Term 1 election: Requested vote from peers c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129:42745), 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263)
I20260811 06:23:08.137008 8652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb7415e3ad48a287ebf65ff1f20985" candidate_uuid: "62b9f74de33b4d408c6494e3e20638c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f"
I20260811 06:23:08.137267 8652 raft_consensus.cc:3037] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:08.137321 8805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb7415e3ad48a287ebf65ff1f20985" candidate_uuid: "62b9f74de33b4d408c6494e3e20638c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2455268dce114057a0cef8178b9df052"
I20260811 06:23:08.137537 8805 raft_consensus.cc:3037] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:08.140565 8805 raft_consensus.cc:2454] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62b9f74de33b4d408c6494e3e20638c4 in term 1.
I20260811 06:23:08.141033 8652 raft_consensus.cc:2454] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62b9f74de33b4d408c6494e3e20638c4 in term 1.
I20260811 06:23:08.141690 8879 leader_election.cc:302] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2455268dce114057a0cef8178b9df052, 62b9f74de33b4d408c6494e3e20638c4; no voters:
I20260811 06:23:08.142017 9034 raft_consensus.cc:2781] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:08.142874 9034 raft_consensus.cc:686] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 LEADER]: Becoming Leader. State: Replica: 62b9f74de33b4d408c6494e3e20638c4, State: Running, Role: LEADER
I20260811 06:23:08.143553 9034 consensus_queue.cc:237] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } }
I20260811 06:23:08.145820 9073 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: TxnStatusTablet state changed. Reason: New leader 62b9f74de33b4d408c6494e3e20638c4. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.146474 9073 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:08.148844 8472 catalog_manager.cc:5495] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131). New cstate: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } health_report { overall_health: HEALTHY } } }
I20260811 06:23:08.154654 9075 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:23:08.154945 9075 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:23:08.206490 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.198149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33687 (local address 127.0.0.1:60776)
0811 06:23:08.198350 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:08.198371 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:08.198508 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:08.199771 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:08.199778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:08.199795 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:08.200023 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.200035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.201686 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.201695 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.203397 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.203413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.204425 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.204431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.204654 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.204728 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.205863 (+ 1135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.205884 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 06:23:08.205980 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:23:08.206141 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:23:08.206552 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.198123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.7.4.131:33687)
0811 06:23:08.199333 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.199339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.199362 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:23:08.199454 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.199460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.199563 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.199718 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:08.200221 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.201483 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.203550 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.204256 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.204387 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.204979 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.205738 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.205882 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.206042 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.206132 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":859,"threads_started":1}
I20260811 06:23:08.212113 8805 raft_consensus.cc:1261] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Refusing update from remote peer 62b9f74de33b4d408c6494e3e20638c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:08.212116 8652 raft_consensus.cc:1261] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Refusing update from remote peer 62b9f74de33b4d408c6494e3e20638c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:08.213146 9034 consensus_queue.cc:1035] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:08.213513 9073 consensus_queue.cc:1035] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:08.228313 9040 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: TxnStatusTablet state changed. Reason: New leader 62b9f74de33b4d408c6494e3e20638c4. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.228785 9040 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:08.232661 9046 mvcc.cc:205] Tried to move back new op lower bound from 7317214774108188672 to 7317214773840175104. Current Snapshot: MvccSnapshot[applied={T|T < 7317214774108188672}]
I20260811 06:23:08.242115 9040 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.242457 9040 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P c2e64fc5e9e14d89b5ddc5475e3e592f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:08.246412 9034 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.246781 9034 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:08.252035 9035 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: TxnStatusTablet state changed. Reason: New leader 62b9f74de33b4d408c6494e3e20638c4. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.252353 9035 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:08.262318 9083 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.262844 9083 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 62b9f74de33b4d408c6494e3e20638c4: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:08.267894 9035 tablet_replica.cc:439] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62b9f74de33b4d408c6494e3e20638c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2e64fc5e9e14d89b5ddc5475e3e592f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42745 } } peers { permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 } } peers { permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 } } }
I20260811 06:23:08.268204 9035 tablet_replica.cc:442] T 31eb7415e3ad48a287ebf65ff1f20985 P 2455268dce114057a0cef8178b9df052: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:08.271703 9040 consensus_queue.cc:1035] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2455268dce114057a0cef8178b9df052" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:08.286679 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.267886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33687 (local address 127.7.4.129:57453)
0811 06:23:08.278217 (+ 10331us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:08.278246 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:08.278676 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:08.279420 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:08.279429 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:08.279448 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:08.279945 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.279960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.281595 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.281600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.283917 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.283930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.285212 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.285220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.285451 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.285529 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.286001 (+ 472us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.286014 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:23:08.286099 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 06:23:08.286268 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10049,"thread_start_us":10502,"threads_started":1}
I20260811 06:23:08.286671 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.269042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:57453 (local address 127.7.4.131:33687)
0811 06:23:08.269213 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.269221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.278360 (+ 9139us) server_negotiation.cc:407] Connection header received
0811 06:23:08.278892 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.278901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.279025 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.279104 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:08.280163 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.281454 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.284060 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.284904 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.285031 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.285747 (+ 716us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.285887 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.285980 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.286156 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.286253 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 06:23:08.287184 9040 consensus_queue.cc:1035] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [LEADER]: Connected to new peer: Peer: permanent_uuid: "62b9f74de33b4d408c6494e3e20638c4" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 06:23:08.346686 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.339871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:48571 (local address 127.7.4.129:42745)
0811 06:23:08.340053 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.340058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.340733 (+ 675us) server_negotiation.cc:407] Connection header received
0811 06:23:08.341006 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.341012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.341103 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.341200 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:08.341851 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.342827 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.344627 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.345261 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.345423 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.345863 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.346035 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.346138 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.346295 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.346396 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:23:08.346895 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.339877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42745 (local address 127.7.4.131:48571)
0811 06:23:08.340607 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:08.340623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:08.340808 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:08.341225 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:08.341229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:08.341242 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:08.341696 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.341711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.342961 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.342968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.344476 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:08.344488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.345465 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.345473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:08.345635 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.345689 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:08.346143 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:08.346157 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 06:23:08.346244 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:23:08.346420 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":419,"threads_started":1}
I20260811 06:23:08.379895 8472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:08.380144 8472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:08.383162 8472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a3c064dc719413089e8d6868ab517b6: Sending DeleteTablet for 3 replicas of tablet 74de356281a9416cb11bc144ec6afd16
I20260811 06:23:08.383833 8624 tablet_service.cc:1514] Processing DeleteTablet for tablet 74de356281a9416cb11bc144ec6afd16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:37136
I20260811 06:23:08.384375 8779 tablet_service.cc:1514] Processing DeleteTablet for tablet 74de356281a9416cb11bc144ec6afd16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:39324
I20260811 06:23:08.384965 8937 tablet_service.cc:1514] Processing DeleteTablet for tablet 74de356281a9416cb11bc144ec6afd16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60768
I20260811 06:23:08.390467 9097 tablet_replica.cc:330] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: stopping tablet replica
I20260811 06:23:08.391557 9097 raft_consensus.cc:2229] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:08.392270 9097 raft_consensus.cc:2258] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:08.394033 9097 ts_tablet_manager.cc:1905] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:08.396260 9094 tablet_replica.cc:330] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: stopping tablet replica
I20260811 06:23:08.396615 9094 raft_consensus.cc:2229] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:08.397051 9094 raft_consensus.cc:2258] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:08.398244 9094 ts_tablet_manager.cc:1905] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:08.398357 9095 tablet_replica.cc:330] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: stopping tablet replica
I20260811 06:23:08.398718 9095 raft_consensus.cc:2229] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:08.399096 9095 raft_consensus.cc:2258] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:08.400345 9095 ts_tablet_manager.cc:1905] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:08.405575 9094 ts_tablet_manager.cc:1918] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:08.406476 9094 log.cc:1196] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/74de356281a9416cb11bc144ec6afd16
I20260811 06:23:08.408179 9097 ts_tablet_manager.cc:1918] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:08.408221 9095 ts_tablet_manager.cc:1918] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:08.408345 9095 log.cc:1196] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/74de356281a9416cb11bc144ec6afd16
I20260811 06:23:08.408625 9097 log.cc:1196] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/74de356281a9416cb11bc144ec6afd16
I20260811 06:23:08.408828 9094 ts_tablet_manager.cc:1939] T 74de356281a9416cb11bc144ec6afd16 P c2e64fc5e9e14d89b5ddc5475e3e592f: Deleting consensus metadata
I20260811 06:23:08.409075 9095 ts_tablet_manager.cc:1939] T 74de356281a9416cb11bc144ec6afd16 P 2455268dce114057a0cef8178b9df052: Deleting consensus metadata
I20260811 06:23:08.409350 9097 ts_tablet_manager.cc:1939] T 74de356281a9416cb11bc144ec6afd16 P 62b9f74de33b4d408c6494e3e20638c4: Deleting consensus metadata
I20260811 06:23:08.409989 8456 catalog_manager.cc:4841] TS c2e64fc5e9e14d89b5ddc5475e3e592f (127.7.4.129:42745): tablet 74de356281a9416cb11bc144ec6afd16 (table test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]) successfully deleted
I20260811 06:23:08.410243 8458 catalog_manager.cc:4841] TS 2455268dce114057a0cef8178b9df052 (127.7.4.130:36263): tablet 74de356281a9416cb11bc144ec6afd16 (table test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]) successfully deleted
I20260811 06:23:08.410497 8458 catalog_manager.cc:4841] TS 62b9f74de33b4d408c6494e3e20638c4 (127.7.4.131:33687): tablet 74de356281a9416cb11bc144ec6afd16 (table test-table [id=52ec43afb3e24712ac7a4ca4b14f8ee8]) successfully deleted
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:08.426680 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.398273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.7.4.129:42745)
0811 06:23:08.398462 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.398468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.406197 (+ 7729us) server_negotiation.cc:407] Connection header received
0811 06:23:08.406766 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.406773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.406856 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.407019 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:08.411362 (+ 4343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.412180 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.414056 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.414521 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.414619 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.419726 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:08.419742 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:08.419759 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:08.419799 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:08.423715 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.424440 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.424449 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.424453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.424636 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.424982 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.424988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.424991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.425467 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:08.425939 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.426107 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.426328 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:23:08.427524 8695 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38196
I20260811 06:23:08.445091 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.430856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.7.4.190:42597)
0811 06:23:08.431492 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.431498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.431821 (+ 323us) server_negotiation.cc:407] Connection header received
0811 06:23:08.431943 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.431948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.432049 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.432177 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:08.433572 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.434690 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.435957 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.436463 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.436598 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.438837 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:08.438842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:08.438845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:08.438875 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:08.442067 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.442826 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.442833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.442837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.442972 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.443401 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.443407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.443410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.443683 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:08.444008 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.444287 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.444631 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":291,"threads_started":1}
W20260811 06:23:08.445786 8472 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51446
I20260811 06:23:08.447641 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8552
I20260811 06:23:08.464094 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.449459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.7.4.129:42745)
0811 06:23:08.449948 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.449956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.451735 (+ 1779us) server_negotiation.cc:407] Connection header received
0811 06:23:08.451918 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.451922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.451981 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.452076 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:08.453179 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.453844 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.455336 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.455943 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.456085 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.458576 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:08.458581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:08.458584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:08.458616 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:08.461428 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.462046 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.462053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.462057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.462202 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.462564 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.462570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.462574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.462882 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:08.463131 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.463500 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.463728 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
W20260811 06:23:08.464669 8695 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38212
W20260811 06:23:08.465193 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8552: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:08.465448 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8552
I20260811 06:23:08.495267 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8706
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:08.512467 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.496143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.7.4.130:36263)
0811 06:23:08.496330 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.496335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.496358 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:23:08.496502 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.496509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.496608 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.496728 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:08.497982 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.499156 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.500512 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.501145 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.501294 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.505435 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:08.505453 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:08.505469 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:08.505519 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:08.509409 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.510063 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.510073 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.510076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.510253 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.510622 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.510627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.510631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.511142 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:08.511488 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.511800 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.512064 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 06:23:08.513150 8850 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49946
W20260811 06:23:08.513847 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8706: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:08.514016 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8706
I20260811 06:23:08.537276 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8861
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[8425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429387, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:08.555915 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:08.538323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.7.4.131:33687)
0811 06:23:08.538523 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:08.538529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:08.538551 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:23:08.538649 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:08.538653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:08.538719 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:08.538842 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:08.540207 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.541009 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.542220 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:08.542732 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:08.542877 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:08.547279 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:08.547314 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:08.547334 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:08.547376 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:08.551824 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.552893 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.552905 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.552911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.553104 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:08.553544 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:08.553554 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:08.553559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:08.554084 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:08.554493 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:08.554886 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 06:23:08.555489 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 06:23:08.556806 9001 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47524
W20260811 06:23:08.558099 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8861: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:08.558274 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8861
I20260811 06:23:08.595369 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 8438
I20260811 06:23:08.596856 8515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:08.715556 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 8438
2026-08-11T06:23:08Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5062 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[9106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[9106](info): set up 2 sockets
Aug 11 06:23:08 dist-test-slave-07c2 krb5kdc[9106](info): commencing operation
krb5kdc: starting...
W20260811 06:23:09.856089 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:09 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:10.064584 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45515
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:10.412124 9119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:10.412431 9119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:10.412593 9119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:10.420244 9119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:10.420380 9119 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:23:10.420470 9119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:10.420583 9119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:10.420665 9119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:10.420782 9119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:10.430859 9119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45515
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:10.432298 9119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:10.433714 9119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:10.445832 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:10.446003 9127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:10.445832 9125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:10.447060 9119 server_base.cc:1034] running on GCE node
I20260811 06:23:10.447942 9119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:10.449443 9119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:10.450706 9119 hybrid_clock.cc:648] HybridClock initialized: now 1786429390450630 us; error 47 us; skew 500 ppm
I20260811 06:23:10.451086 9119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:10 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429390, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:10.454283 9119 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:10.455905 9119 webserver.cc:460] Webserver started at http://127.7.4.190:38127/ using document root <none> and password file <none>
I20260811 06:23:10.456635 9119 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:10.456774 9119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:10.457118 9119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:10.460268 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "3e104e610c0e45fb8d14f77524763748"
format_stamp: "Formatted at 2026-08-11 06:23:10 on dist-test-slave-07c2"
server_key: "40c153f3ba8c16ee0f79efc9c39b7df4"
server_key_iv: "a3f17d4ab2e21a48ce9a941cb4b0c2dd"
server_key_version: "encryptionkey@0"
I20260811 06:23:10.461150 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "3e104e610c0e45fb8d14f77524763748"
format_stamp: "Formatted at 2026-08-11 06:23:10 on dist-test-slave-07c2"
server_key: "40c153f3ba8c16ee0f79efc9c39b7df4"
server_key_iv: "a3f17d4ab2e21a48ce9a941cb4b0c2dd"
server_key_version: "encryptionkey@0"
I20260811 06:23:10.466634 9119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:23:10.470709 9135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:10.472158 9119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:10.472357 9119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "3e104e610c0e45fb8d14f77524763748"
format_stamp: "Formatted at 2026-08-11 06:23:10 on dist-test-slave-07c2"
server_key: "40c153f3ba8c16ee0f79efc9c39b7df4"
server_key_iv: "a3f17d4ab2e21a48ce9a941cb4b0c2dd"
server_key_version: "encryptionkey@0"
I20260811 06:23:10.472592 9119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:10.490651 9119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:10.495201 9119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:10.495525 9119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:10.518422 9119 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45515
I20260811 06:23:10.518443 9202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45515 every 8 connection(s)
I20260811 06:23:10.519990 9119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:10.523874 9203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:10.524638 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9119
I20260811 06:23:10.524896 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:10.525772 7186 external_mini_cluster.cc:1428] Setting key 6aeb79d990a63cc42553c5e3e9b157de
I20260811 06:23:10.539558 9203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Bootstrap starting.
Aug 11 06:23:10 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:10.543566 9203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:10.544849 9203 log.cc:824] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:10.548329 9203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: No bootstrap required, opened a new log
I20260811 06:23:10.550688 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:10.528050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36750 (local address 127.7.4.190:45515)
0811 06:23:10.528958 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:10.528973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:10.529018 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:23:10.530780 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:10.530803 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:10.531361 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:10.531728 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:10.533650 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:10.535033 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:10.536903 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:10.537621 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:10.537758 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:10.542748 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:10.542759 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:10.542782 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:10.542840 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:10.546397 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:10.547268 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:10.547274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:10.547280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:10.547503 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:10.547935 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:10.547939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:10.547941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:10.548501 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:10.548744 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:10.549248 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 06:23:10.549588 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":682,"thread_start_us":440,"threads_started":1}
I20260811 06:23:10.556478 9203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } }
I20260811 06:23:10.556877 9203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:10.556988 9203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e104e610c0e45fb8d14f77524763748, State: Initialized, Role: FOLLOWER
I20260811 06:23:10.557602 9203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } }
I20260811 06:23:10.557863 9203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:10.558015 9203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:10.558202 9203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:10.560554 9203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } }
I20260811 06:23:10.561062 9203 leader_election.cc:302] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e104e610c0e45fb8d14f77524763748; no voters:
I20260811 06:23:10.561641 9203 leader_election.cc:288] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:10.561800 9209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:10.562637 9209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [term 1 LEADER]: Becoming Leader. State: Replica: 3e104e610c0e45fb8d14f77524763748, State: Running, Role: LEADER
I20260811 06:23:10.563176 9209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } }
I20260811 06:23:10.563834 9203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:10.565996 9211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e104e610c0e45fb8d14f77524763748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } } }
I20260811 06:23:10.566390 9211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:10.566883 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e104e610c0e45fb8d14f77524763748. Latest consensus state: current_term: 1 leader_uuid: "3e104e610c0e45fb8d14f77524763748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e104e610c0e45fb8d14f77524763748" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45515 } } }
I20260811 06:23:10.567195 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:10.568437 9216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:10.574363 9216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:10.583622 9216 catalog_manager.cc:1347] Generated new cluster ID: 6a0e215acb4c46a2a1885a3acb6a57eb
I20260811 06:23:10.583863 9216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:10.595408 9216 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:10.596858 9216 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:10.606832 9216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Generated new TSK 0
I20260811 06:23:10.607937 9216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:10.676949 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45515
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:11.006755 9232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:11.007027 9232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:11.007203 9232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:11.014386 9232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:11.014515 9232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:11.014690 9232 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:23:11.014775 9232 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:11.014907 9232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:11.024660 9232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:11.025923 9232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:11.027346 9232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:11.040098 9237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:11.040068 9240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:11.040158 9238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:11.040714 9232 server_base.cc:1034] running on GCE node
I20260811 06:23:11.041352 9232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:11.042198 9232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:11.043390 9232 hybrid_clock.cc:648] HybridClock initialized: now 1786429391043337 us; error 53 us; skew 500 ppm
I20260811 06:23:11.043696 9232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:11 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:11.051744 9232 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:11.053263 9232 webserver.cc:460] Webserver started at http://127.7.4.129:33075/ using document root <none> and password file <none>
I20260811 06:23:11.053943 9232 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:11.054093 9232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:11.054417 9232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:11.057510 9232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "86b4b5f0e05ab34cd12717c46afc007b"
server_key_iv: "42ceffcaaf30790a96a8ccb5059d9256"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.058298 9232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "86b4b5f0e05ab34cd12717c46afc007b"
server_key_iv: "42ceffcaaf30790a96a8ccb5059d9256"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.063251 9232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:11.066742 9248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:11.068140 9232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:11.068307 9232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "86b4b5f0e05ab34cd12717c46afc007b"
server_key_iv: "42ceffcaaf30790a96a8ccb5059d9256"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.068506 9232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:11.091841 9232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:11.095712 9232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:11.096004 9232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:11.098661 9232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:11.098809 9232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:11.098919 9232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:11.099027 9232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:11 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:11.124109 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.098995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:47899 (local address 127.7.4.190:45515)
0811 06:23:11.099540 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:11.099546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:11.100732 (+ 1186us) server_negotiation.cc:407] Connection header received
0811 06:23:11.103599 (+ 2867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:11.103607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:11.103715 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:11.103847 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:11.106331 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.107216 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.110459 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.110988 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.111141 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.116193 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:11.116202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:11.116210 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:11.116262 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:11.120270 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.121285 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.121289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.121291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.121404 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.122016 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.122024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.122030 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.122428 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:11.122580 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:11.123513 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 06:23:11.123743 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":227,"threads_started":1}
I20260811 06:23:11.125037 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.099650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.129:47899)
0811 06:23:11.100541 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:11.100595 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:11.103249 (+ 2654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:11.104154 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:11.104164 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:11.105114 (+ 950us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:11.106103 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.106143 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.107419 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.107428 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.110261 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.110304 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.111318 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.111330 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.111709 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.112578 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:11.112619 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:11.115929 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:11.120455 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.120466 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.120481 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.121087 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.121572 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.121579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.121584 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.121900 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.122548 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:11.122554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:11.123031 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 06:23:11.123419 (+ 388us) client_negotiation.cc:240] Negotiation successful
0811 06:23:11.123885 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":318,"threads_started":1}
I20260811 06:23:11.143797 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.130857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.129:42405)
0811 06:23:11.131880 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:11.131901 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:11.132055 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:11.132648 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:11.132654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:11.133291 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:11.133793 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.133812 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.134770 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.134777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.136143 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.136157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.136878 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.136884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.137052 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.137621 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:11.137644 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:11.138352 (+ 708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:11.141295 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.141302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.141306 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.141699 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.142074 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.142079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.142083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.142209 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.142714 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:11.142719 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:11.142906 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 06:23:11.143134 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 06:23:11.143478 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":618,"threads_started":1}
I20260811 06:23:11.144089 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.130732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:42405 (local address 127.7.4.190:45515)
0811 06:23:11.130922 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:11.130929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:11.131980 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 06:23:11.132247 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:11.132251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:11.132328 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:11.132403 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:11.133943 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.134613 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.136285 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.136677 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.136812 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.138538 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:11.138542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:11.138545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:11.138573 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:11.141146 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.141813 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.141820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.141824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.141954 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.142318 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.142324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.142328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.142597 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:11.142749 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:11.143302 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 06:23:11.143599 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:23:11.157804 9232 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:38279
I20260811 06:23:11.157845 9380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:38279 every 8 connection(s)
I20260811 06:23:11.159092 9232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:11.161188 9381 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45515
I20260811 06:23:11.161372 9381 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:11.161916 9381 heartbeater.cc:511] Master 127.7.4.190:45515 requested a full tablet report, sending...
I20260811 06:23:11.163944 9152 ts_manager.cc:194] Registered new tserver with Master: af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279)
I20260811 06:23:11.165469 9152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:47899
I20260811 06:23:11.169024 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9232
I20260811 06:23:11.169240 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:11.169978 7186 external_mini_cluster.cc:1428] Setting key ac9e9fdaca709966fb0d3dee40d62a51
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:11.214505 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45515
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:11.544402 9385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:11.544678 9385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:11.544785 9385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:11.552134 9385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:11.552238 9385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:11.552323 9385 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:23:11.552377 9385 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:11.552452 9385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:11.562111 9385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:11.563421 9385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:11.564940 9385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:11.578759 9391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:11.579041 9390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:11.579602 9394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:11.579705 9385 server_base.cc:1034] running on GCE node
I20260811 06:23:11.580451 9385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:11.581653 9385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:11.582870 9385 hybrid_clock.cc:648] HybridClock initialized: now 1786429391582768 us; error 86 us; skew 500 ppm
I20260811 06:23:11.583238 9385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:11 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:11.586391 9385 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:11.587934 9385 webserver.cc:460] Webserver started at http://127.7.4.130:39633/ using document root <none> and password file <none>
I20260811 06:23:11.588553 9385 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:11.588665 9385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:11.588924 9385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:11.591846 9385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "51edfb55467f48138c512d78a0081bdf"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "5dba7c521d429f339b0072d228492bfe"
server_key_iv: "9b3ef513bdb7368b82ccb8f921e89554"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.592530 9385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "51edfb55467f48138c512d78a0081bdf"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "5dba7c521d429f339b0072d228492bfe"
server_key_iv: "9b3ef513bdb7368b82ccb8f921e89554"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.597496 9385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:23:11.600857 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:11.602140 9385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:11.602305 9385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "51edfb55467f48138c512d78a0081bdf"
format_stamp: "Formatted at 2026-08-11 06:23:11 on dist-test-slave-07c2"
server_key: "5dba7c521d429f339b0072d228492bfe"
server_key_iv: "9b3ef513bdb7368b82ccb8f921e89554"
server_key_version: "encryptionkey@0"
I20260811 06:23:11.602497 9385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:11.615437 9385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:11.619621 9385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:11.619907 9385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:11.622452 9385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:11.622713 9385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:11.622900 9385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:11.623039 9385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:11 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429391, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:11.651213 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.623268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:42947 (local address 127.7.4.190:45515)
0811 06:23:11.623519 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:11.623526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:11.624825 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 06:23:11.626903 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:11.626909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:11.627005 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:11.627117 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:11.629334 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.630450 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.633086 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.633865 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.634001 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.639360 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:11.639370 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:11.639378 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:11.639433 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:11.643985 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.646732 (+ 2747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.646740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.646746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.646937 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.648031 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.648038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.648043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.648477 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:11.648679 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:11.650432 (+ 1753us) server_negotiation.cc:299] Negotiation successful
0811 06:23:11.650721 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 06:23:11.652921 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.623863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.130:42947)
0811 06:23:11.624659 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:11.624705 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:11.626638 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:11.627401 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:11.627409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:11.628186 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:11.629083 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.629113 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.630638 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.630642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.632840 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.632877 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.634139 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.634151 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.634371 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.635058 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:11.635080 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:11.639017 (+ 3937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:11.645664 (+ 6647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.645677 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.645704 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.646561 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.647469 (+ 908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.647474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.647480 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.647810 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.649254 (+ 1444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:11.649262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:11.649849 (+ 587us) client_negotiation.cc:769] Sending connection context
0811 06:23:11.650247 (+ 398us) client_negotiation.cc:240] Negotiation successful
0811 06:23:11.650964 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":364,"threads_started":1}
I20260811 06:23:11.674191 9385 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34007
I20260811 06:23:11.674243 9535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34007 every 8 connection(s)
I20260811 06:23:11.674885 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.659031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:33283 (local address 127.7.4.190:45515)
0811 06:23:11.659244 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:11.659253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:11.660068 (+ 815us) server_negotiation.cc:407] Connection header received
0811 06:23:11.660264 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:11.660267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:11.660326 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:11.660444 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:11.661939 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.663370 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.665135 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.665812 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.665959 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.667988 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:11.667995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:11.667999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:11.668037 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:11.672014 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.672692 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.672700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.672706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.672872 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:11.673345 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:11.673351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:11.673357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:11.673690 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:11.673860 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:11.674311 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 06:23:11.674534 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:23:11.675024 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:11.659040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.130:33283)
0811 06:23:11.659943 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:11.659968 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:11.660140 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:11.660485 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:11.660492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:11.661230 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:11.661752 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.661767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.663533 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.663544 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.664932 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:11.664950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:11.666027 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:11.666034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:11.666215 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:11.666986 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:11.667003 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:11.667776 (+ 773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:11.672149 (+ 4373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.672154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.672158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.672532 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.673005 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:11.673010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:11.673013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:11.673186 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:11.673829 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:11.673834 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:11.674040 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 06:23:11.674216 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:23:11.674565 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":469,"threads_started":1}
I20260811 06:23:11.675768 9385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:11.678457 9536 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45515
I20260811 06:23:11.678682 9536 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:11.679369 9536 heartbeater.cc:511] Master 127.7.4.190:45515 requested a full tablet report, sending...
I20260811 06:23:11.680685 9152 ts_manager.cc:194] Registered new tserver with Master: 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007)
I20260811 06:23:11.681502 9152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:42947
I20260811 06:23:11.683096 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9385
I20260811 06:23:11.683339 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:11.683885 7186 external_mini_cluster.cc:1428] Setting key 779056783768b519b12a58f8026301d4
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:11.730227 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45515
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:12.056113 9541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:12.056471 9541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:12.056679 9541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:12.063601 9541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:12.063740 9541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:12.063892 9541 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:23:12.063977 9541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:23:12.064098 9541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:12.073824 9541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:12.075088 9541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:12.076647 9541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:12.089691 9546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:12.089651 9550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:12.089763 9547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:12.090457 9541 server_base.cc:1034] running on GCE node
I20260811 06:23:12.091095 9541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:12.092099 9541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:12.093302 9541 hybrid_clock.cc:648] HybridClock initialized: now 1786429392093257 us; error 35 us; skew 500 ppm
I20260811 06:23:12.093611 9541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:12 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:12.097009 9541 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:12.098410 9541 webserver.cc:460] Webserver started at http://127.7.4.131:39797/ using document root <none> and password file <none>
I20260811 06:23:12.099031 9541 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:12.099177 9541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:12.099535 9541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:12.102366 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
format_stamp: "Formatted at 2026-08-11 06:23:12 on dist-test-slave-07c2"
server_key: "268558111ec620b9f9f886608a92b069"
server_key_iv: "9119f87613df3e630c3a77a2640d28b5"
server_key_version: "encryptionkey@0"
I20260811 06:23:12.103116 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
format_stamp: "Formatted at 2026-08-11 06:23:12 on dist-test-slave-07c2"
server_key: "268558111ec620b9f9f886608a92b069"
server_key_iv: "9119f87613df3e630c3a77a2640d28b5"
server_key_version: "encryptionkey@0"
I20260811 06:23:12.108568 9541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:23:12.111761 9558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:12.112972 9541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260811 06:23:12.113152 9541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
format_stamp: "Formatted at 2026-08-11 06:23:12 on dist-test-slave-07c2"
server_key: "268558111ec620b9f9f886608a92b069"
server_key_iv: "9119f87613df3e630c3a77a2640d28b5"
server_key_version: "encryptionkey@0"
I20260811 06:23:12.113360 9541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:12.129788 9541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:12.133651 9541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:12.133939 9541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:12.136781 9541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:12.136922 9541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:12.137068 9541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:12.137183 9541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:23:12 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429392, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:12.161516 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.137029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:50205 (local address 127.7.4.190:45515)
0811 06:23:12.137266 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.137273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.138622 (+ 1349us) server_negotiation.cc:407] Connection header received
0811 06:23:12.140829 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.140835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.140935 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.141084 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:12.143241 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.144693 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.147488 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.148291 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.148456 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.154885 (+ 6429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:12.154896 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:12.154904 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:12.154955 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:12.158201 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.159112 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.159117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.159125 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.159250 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.159788 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.159798 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.159800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.160078 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:12.160248 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.160918 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.161167 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:23:12.162500 9567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.137569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.131:50205)
0811 06:23:12.138440 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.138484 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.140497 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.141352 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.141363 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.142160 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:12.143002 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.143050 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.144934 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.144941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.147239 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.147253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.148883 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.148894 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.149298 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.150238 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:12.150277 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:12.154580 (+ 4303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:12.158346 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.158359 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.158377 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.158975 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.159391 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.159397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.159401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.159658 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.160219 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:12.160227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:12.160559 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.160816 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.161332 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":374,"threads_started":1}
I20260811 06:23:12.168064 9381 heartbeater.cc:503] Master 127.7.4.190:45515 was elected leader, sending a full tablet report...
I20260811 06:23:12.184422 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.168377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:55803 (local address 127.7.4.190:45515)
0811 06:23:12.168588 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.168593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.169789 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 06:23:12.170032 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.170037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.170112 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.170212 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:12.171186 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.171934 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.173485 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.174472 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.174652 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.177538 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:12.177558 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:12.177567 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:12.177610 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:12.181269 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.182064 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.182070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.182073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.182264 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.182824 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.182832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.182836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.183218 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:12.183421 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.183827 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.184046 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:23:12.184513 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.168285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.7.4.131:55803)
0811 06:23:12.169694 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.169713 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.169893 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.170241 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.170245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.170763 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:12.171047 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.171056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.172070 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.172077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.173207 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.173225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.174729 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.174734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.174901 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.175738 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:12.175769 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:12.177200 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:12.181395 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.181401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.181405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.181919 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.182434 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.182440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.182445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.182668 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.183380 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:12.183385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:12.183578 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.183730 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.184048 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":1130,"threads_started":1}
I20260811 06:23:12.199540 9541 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:36905
I20260811 06:23:12.199632 9690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:36905 every 8 connection(s)
I20260811 06:23:12.200775 9541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:12.202808 9691 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45515
I20260811 06:23:12.203042 9691 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:12.203734 9691 heartbeater.cc:511] Master 127.7.4.190:45515 requested a full tablet report, sending...
I20260811 06:23:12.205013 9152 ts_manager.cc:194] Registered new tserver with Master: fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:12.205824 9152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:50205
I20260811 06:23:12.209105 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9541
I20260811 06:23:12.209266 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:12.209661 7186 external_mini_cluster.cc:1428] Setting key 0caf723b34ec0a93d3d2ac4aa0b89a43
I20260811 06:23:12.212576 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:23:12 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:23:12 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:12.266517 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.252585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.7.4.190:45515)
0811 06:23:12.252841 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.252848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.253104 (+ 256us) server_negotiation.cc:407] Connection header received
0811 06:23:12.253263 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.253268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.253336 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.253422 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:12.254784 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.255638 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.256892 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.257308 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.257396 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.261129 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:12.261142 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:12.261145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:12.261176 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:12.263671 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.264394 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.264398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.264400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.264572 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.264989 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.264995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.264999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.265330 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:12.265592 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.265895 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.266119 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 06:23:12.270658 9152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:12.277664 9152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:12.311065 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.289562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.7.4.130:34007)
0811 06:23:12.301450 (+ 11888us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.301464 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.303815 (+ 2351us) server_negotiation.cc:407] Connection header received
0811 06:23:12.303937 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.303944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.304166 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.304448 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.305177 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.306601 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.308757 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.309917 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.310053 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.310293 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.310592 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.310760 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11662,"thread_start_us":11520,"threads_started":1}
I20260811 06:23:12.311193 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.289178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34007 (local address 127.0.0.1:43012)
0811 06:23:12.303521 (+ 14343us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.303537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.303678 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.304417 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.304421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.304446 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.304992 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.305005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.306790 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.306795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.308573 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.308609 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.310121 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.310136 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.310367 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.310415 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.310547 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.310720 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14205,"mutex_wait_us":72,"thread_start_us":15640,"threads_started":1}
I20260811 06:23:12.312077 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.288913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.0.0.1:36530)
0811 06:23:12.301930 (+ 13017us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.301987 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.302179 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.303525 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.303536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.303580 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.304133 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.304149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.306697 (+ 2548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.306701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.309117 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.309129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.311247 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.311253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.311437 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.311481 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.311564 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.311680 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12741,"mutex_wait_us":88,"thread_start_us":12534,"threads_started":1}
I20260811 06:23:12.312181 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.289200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.7.4.129:38279)
0811 06:23:12.302712 (+ 13512us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.302724 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.302762 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:23:12.302890 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.302901 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.303209 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.304466 (+ 1257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.304733 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.306411 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.309268 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.310769 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.310894 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.311089 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.311748 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.311896 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13315,"thread_start_us":13151,"threads_started":1}
I20260811 06:23:12.315307 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.291839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.7.4.131:36905)
0811 06:23:12.302986 (+ 11147us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.302998 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.305207 (+ 2209us) server_negotiation.cc:407] Connection header received
0811 06:23:12.305378 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.305407 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.305616 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.305805 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.306438 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.308786 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.311998 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.314243 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.314384 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.314601 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.314819 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.314962 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10938,"thread_start_us":10726,"threads_started":1}
I20260811 06:23:12.315323 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.291023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:36905 (local address 127.0.0.1:35930)
0811 06:23:12.305113 (+ 14090us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.305130 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.305237 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.305795 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.305799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.305826 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.306210 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.306220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.309291 (+ 3071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.309295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.311816 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.311830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.314447 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.314453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.314637 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.314680 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.314766 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.314889 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13958,"mutex_wait_us":4,"thread_start_us":12179,"threads_started":1}
I20260811 06:23:12.315191 9456 tablet_service.cc:1467] Processing CreateTablet for tablet ce4692e725bd46d898a04ae657b43d46 (DEFAULT_TABLE table=test-table [id=f5413e4949be4196a3ae0c7daebea35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.315451 9303 tablet_service.cc:1467] Processing CreateTablet for tablet ce4692e725bd46d898a04ae657b43d46 (DEFAULT_TABLE table=test-table [id=f5413e4949be4196a3ae0c7daebea35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.316741 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4692e725bd46d898a04ae657b43d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.316756 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4692e725bd46d898a04ae657b43d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.318763 9611 tablet_service.cc:1467] Processing CreateTablet for tablet ce4692e725bd46d898a04ae657b43d46 (DEFAULT_TABLE table=test-table [id=f5413e4949be4196a3ae0c7daebea35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.319905 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4692e725bd46d898a04ae657b43d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.331573 9711 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Bootstrap starting.
I20260811 06:23:12.334581 9711 tablet_bootstrap.cc:654] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.335528 9712 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Bootstrap starting.
I20260811 06:23:12.336140 9711 log.cc:824] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:12.337304 9713 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Bootstrap starting.
I20260811 06:23:12.338819 9712 tablet_bootstrap.cc:654] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.339483 9711 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: No bootstrap required, opened a new log
I20260811 06:23:12.339758 9711 ts_tablet_manager.cc:1397] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:23:12.340005 9712 log.cc:824] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:12.341017 9713 tablet_bootstrap.cc:654] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.342176 9713 log.cc:824] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:12.342865 9712 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: No bootstrap required, opened a new log
I20260811 06:23:12.343151 9712 ts_tablet_manager.cc:1397] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:23:12.345095 9713 tablet_bootstrap.cc:492] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: No bootstrap required, opened a new log
I20260811 06:23:12.345361 9713 ts_tablet_manager.cc:1397] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:23:12.350227 9712 raft_consensus.cc:348] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.350360 9711 raft_consensus.cc:348] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.350656 9712 raft_consensus.cc:374] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.350750 9711 raft_consensus.cc:374] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.350764 9712 raft_consensus.cc:729] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.350845 9711 raft_consensus.cc:729] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.351475 9712 consensus_queue.cc:260] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.351496 9711 consensus_queue.cc:260] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.352589 9536 heartbeater.cc:503] Master 127.7.4.190:45515 was elected leader, sending a full tablet report...
I20260811 06:23:12.353067 9712 ts_tablet_manager.cc:1428] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20260811 06:23:12.353732 9691 heartbeater.cc:503] Master 127.7.4.190:45515 was elected leader, sending a full tablet report...
I20260811 06:23:12.355099 9711 ts_tablet_manager.cc:1428] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 06:23:12.355711 9713 raft_consensus.cc:348] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.356138 9713 raft_consensus.cc:374] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.356261 9713 raft_consensus.cc:729] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.356841 9713 consensus_queue.cc:260] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.358470 9713 ts_tablet_manager.cc:1428] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 06:23:12.410722 9717 raft_consensus.cc:482] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:12.411022 9717 raft_consensus.cc:504] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
W20260811 06:23:12.411535 9382 tablet.cc:2369] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:12.412704 9717 leader_election.cc:288] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:12.419719 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.413087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:47117 (local address 127.7.4.129:38279)
0811 06:23:12.413311 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.413317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.414189 (+ 872us) server_negotiation.cc:407] Connection header received
0811 06:23:12.414410 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.414422 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.414492 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.414591 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.415166 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.415949 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.417878 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.418710 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.418868 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.418976 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.419340 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.419462 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:23:12.419865 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.413088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:35895 (local address 127.7.4.131:36905)
0811 06:23:12.413306 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.413311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.414355 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 06:23:12.414560 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.414564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.414629 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.414728 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.415400 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.416093 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.418086 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.418841 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.418992 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.419114 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.419483 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.419606 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:23:12.419885 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.413088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.7.4.130:47117)
0811 06:23:12.414066 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.414089 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.414270 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.414611 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.414616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.414642 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.415015 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.415025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.416086 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.416094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.417734 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.417746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.418936 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.418942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.419159 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.419184 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.419292 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.419438 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":28,"thread_start_us":608,"threads_started":1}
I20260811 06:23:12.420006 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.413300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:36905 (local address 127.7.4.130:35895)
0811 06:23:12.414245 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.414259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.414404 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.414752 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.414757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.414795 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.415210 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.415219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.416258 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.416263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.417920 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.417932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.419098 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.419105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.419276 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.419349 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.419455 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.419575 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":52,"thread_start_us":581,"threads_started":1}
I20260811 06:23:12.420982 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4692e725bd46d898a04ae657b43d46" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" is_pre_election: true
I20260811 06:23:12.420984 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4692e725bd46d898a04ae657b43d46" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" is_pre_election: true
I20260811 06:23:12.421520 9332 raft_consensus.cc:2454] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:12.421520 9641 raft_consensus.cc:2454] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:12.422248 9408 leader_election.cc:302] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:12.422663 9717 raft_consensus.cc:2781] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:12.422821 9717 raft_consensus.cc:482] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:12.422961 9717 raft_consensus.cc:3037] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:12.425223 9717 raft_consensus.cc:504] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.425734 9717 leader_election.cc:288] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Requested vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:12.426187 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4692e725bd46d898a04ae657b43d46" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
I20260811 06:23:12.426239 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4692e725bd46d898a04ae657b43d46" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
I20260811 06:23:12.426386 9332 raft_consensus.cc:3037] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:12.426422 9641 raft_consensus.cc:3037] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Advancing to term 1
W20260811 06:23:12.428434 9537 tablet.cc:2369] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:12.428843 9332 raft_consensus.cc:2454] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:12.429231 9641 raft_consensus.cc:2454] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:12.429239 9405 leader_election.cc:302] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, af6626c4d13c4eacbc3b536ba1da0fb7; no voters:
I20260811 06:23:12.429548 9717 raft_consensus.cc:2781] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:12.430040 9717 raft_consensus.cc:686] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Becoming Leader. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Running, Role: LEADER
I20260811 06:23:12.430485 9717 consensus_queue.cc:237] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.435170 9150 catalog_manager.cc:5495] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 51edfb55467f48138c512d78a0081bdf (127.7.4.130). New cstate: current_term: 1 leader_uuid: "51edfb55467f48138c512d78a0081bdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } health_report { overall_health: UNKNOWN } } }
W20260811 06:23:12.452409 9692 tablet.cc:2369] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:23:12 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429390, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:12.515060 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.501914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45515 (local address 127.0.0.1:36768)
0811 06:23:12.502501 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.502520 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.502694 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.503005 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.503009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.503774 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:12.504016 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.504025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.505064 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.505073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.506181 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.506194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.506962 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.506969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.507116 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.507805 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:12.507825 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:12.510045 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:12.512775 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.512785 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.512791 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.513214 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.513525 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:12.513528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:12.513530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:12.513704 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:12.514202 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:12.514208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:12.514360 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.514491 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.514691 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":333,"threads_started":1}
I20260811 06:23:12.515093 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.501892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.7.4.190:45515)
0811 06:23:12.502067 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.502072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.502606 (+ 534us) server_negotiation.cc:407] Connection header received
0811 06:23:12.502813 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.502821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.502876 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.502975 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:12.504159 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.504938 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.506325 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.506744 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.506923 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.510224 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:12.510229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:12.510232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:12.510261 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:12.512660 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.513309 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.513313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.513315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.513433 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:12.513814 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:12.513817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:12.513819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:12.514078 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:12.514230 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.514580 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.514820 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":66}
I20260811 06:23:12.520562 9150 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36768:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:23:12.521837 9150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:12.529029 9303 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae6503439c4261aa3477b3048bb9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2293d1c8a14afd847982140bf8cde1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:12.529477 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae6503439c4261aa3477b3048bb9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2293d1c8a14afd847982140bf8cde1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:12.530191 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae6503439c4261aa3477b3048bb9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.530339 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae6503439c4261aa3477b3048bb9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.537760 9711 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: Bootstrap starting.
I20260811 06:23:12.537993 9456 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae6503439c4261aa3477b3048bb9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab2293d1c8a14afd847982140bf8cde1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:23:12.538753 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae6503439c4261aa3477b3048bb9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.539227 9711 tablet_bootstrap.cc:654] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.540525 9713 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: Bootstrap starting.
I20260811 06:23:12.541350 9711 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: No bootstrap required, opened a new log
I20260811 06:23:12.541452 9711 ts_tablet_manager.cc:1397] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:23:12.542011 9711 raft_consensus.cc:348] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.542196 9711 raft_consensus.cc:374] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.542204 9713 tablet_bootstrap.cc:654] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.542256 9711 raft_consensus.cc:729] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.542400 9711 consensus_queue.cc:260] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.542972 9711 ts_tablet_manager.cc:1428] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 06:23:12.542836 9718 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.543242 9718 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.545068 9713 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: No bootstrap required, opened a new log
I20260811 06:23:12.545267 9713 ts_tablet_manager.cc:1397] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:23:12.545908 9713 raft_consensus.cc:348] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.546144 9713 raft_consensus.cc:374] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.546190 9712 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: Bootstrap starting.
I20260811 06:23:12.546284 9713 raft_consensus.cc:729] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.546547 9713 consensus_queue.cc:260] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.547211 9713 ts_tablet_manager.cc:1428] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:23:12.547015 9719 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.547444 9719 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.548029 9712 tablet_bootstrap.cc:654] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.550693 9712 tablet_bootstrap.cc:492] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: No bootstrap required, opened a new log
I20260811 06:23:12.550843 9712 ts_tablet_manager.cc:1397] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:23:12.551671 9712 raft_consensus.cc:348] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.551848 9712 raft_consensus.cc:374] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.551914 9712 raft_consensus.cc:729] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.552136 9712 consensus_queue.cc:260] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.552760 9712 ts_tablet_manager.cc:1428] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:23:12.552585 9717 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.552919 9717 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.661188 9719 raft_consensus.cc:482] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:12.661443 9719 raft_consensus.cc:504] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.663029 9719 leader_election.cc:288] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:12.670221 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.663491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:40679 (local address 127.7.4.130:34007)
0811 06:23:12.663729 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.663738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.664559 (+ 821us) server_negotiation.cc:407] Connection header received
0811 06:23:12.664713 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.664720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.664830 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.664943 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.665657 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.666507 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.668409 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.669223 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.669327 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.669456 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.669788 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.669933 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:23:12.670306 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.663388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34007 (local address 127.7.4.129:40679)
0811 06:23:12.664428 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.664451 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.664592 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.664964 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.664971 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.665002 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.665489 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.665502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.666636 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.666642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.668247 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.668259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.669452 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.669458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.669629 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.669658 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.669739 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.669874 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":30,"thread_start_us":583,"threads_started":1}
I20260811 06:23:12.670598 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.663898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:46385 (local address 127.7.4.131:36905)
0811 06:23:12.664083 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.664088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.664874 (+ 786us) server_negotiation.cc:407] Connection header received
0811 06:23:12.665117 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.665122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.665205 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.665306 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:12.666008 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.666766 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.668482 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.669272 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.669437 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.669526 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.670036 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.670177 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:12.670836 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.663424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:36905 (local address 127.7.4.129:46385)
0811 06:23:12.664769 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.664785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.664918 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.665328 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.665333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.665360 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:12.665830 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.665841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.666911 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.666915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.668298 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.668334 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.669535 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.669541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.669814 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.669845 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.669977 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.670117 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":71,"spinlock_wait_cycles":36096,"thread_start_us":824,"threads_started":1}
I20260811 06:23:12.671401 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae6503439c4261aa3477b3048bb9ff" candidate_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" is_pre_election: true
I20260811 06:23:12.671269 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae6503439c4261aa3477b3048bb9ff" candidate_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51edfb55467f48138c512d78a0081bdf" is_pre_election: true
I20260811 06:23:12.671663 9641 raft_consensus.cc:2454] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af6626c4d13c4eacbc3b536ba1da0fb7 in term 0.
I20260811 06:23:12.671716 9480 raft_consensus.cc:2454] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af6626c4d13c4eacbc3b536ba1da0fb7 in term 0.
I20260811 06:23:12.672104 9251 leader_election.cc:302] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af6626c4d13c4eacbc3b536ba1da0fb7, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:12.672478 9719 raft_consensus.cc:2781] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:12.672582 9719 raft_consensus.cc:482] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:12.672689 9719 raft_consensus.cc:3037] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:12.674952 9719 raft_consensus.cc:504] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.675549 9719 leader_election.cc:288] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [CANDIDATE]: Term 1 election: Requested vote from peers 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:12.676092 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae6503439c4261aa3477b3048bb9ff" candidate_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51edfb55467f48138c512d78a0081bdf"
I20260811 06:23:12.676319 9480 raft_consensus.cc:3037] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:12.679517 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae6503439c4261aa3477b3048bb9ff" candidate_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
I20260811 06:23:12.679724 9480 raft_consensus.cc:2454] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af6626c4d13c4eacbc3b536ba1da0fb7 in term 1.
I20260811 06:23:12.679785 9641 raft_consensus.cc:3037] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:12.680150 9252 leader_election.cc:302] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, af6626c4d13c4eacbc3b536ba1da0fb7; no voters:
I20260811 06:23:12.680428 9719 raft_consensus.cc:2781] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:12.680876 9719 raft_consensus.cc:686] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 LEADER]: Becoming Leader. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Running, Role: LEADER
I20260811 06:23:12.681404 9719 consensus_queue.cc:237] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.682484 9641 raft_consensus.cc:2454] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af6626c4d13c4eacbc3b536ba1da0fb7 in term 1.
I20260811 06:23:12.685256 9736 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: TxnStatusTablet state changed. Reason: New leader af6626c4d13c4eacbc3b536ba1da0fb7. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.685632 9736 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:12.686381 9738 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:23:12.686661 9738 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:23:12.687384 9150 catalog_manager.cc:5495] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20260811 06:23:12.739012 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.733208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.7.4.129:38279)
0811 06:23:12.733379 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.733383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.733663 (+ 280us) server_negotiation.cc:407] Connection header received
0811 06:23:12.733951 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.733955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.734045 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.734204 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:12.734574 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.735469 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.736755 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.737209 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.737358 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.737825 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.738349 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.738513 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.738578 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.738671 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:23:12.739012 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.733248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.0.0.1:36534)
0811 06:23:12.733549 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.733565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.733739 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.734223 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.734228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.734240 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:12.734428 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.734436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.735631 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.735640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.736603 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.736614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.737375 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.737381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.737551 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.737602 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.738461 (+ 859us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.738483 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.738558 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.738718 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 06:23:12.746564 9480 raft_consensus.cc:1261] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Refusing update from remote peer af6626c4d13c4eacbc3b536ba1da0fb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:12.746754 9641 raft_consensus.cc:1261] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Refusing update from remote peer af6626c4d13c4eacbc3b536ba1da0fb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:12.747519 9736 consensus_queue.cc:1035] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:12.747939 9719 consensus_queue.cc:1035] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:12.764577 9717 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: TxnStatusTablet state changed. Reason: New leader af6626c4d13c4eacbc3b536ba1da0fb7. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.764706 9718 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: TxnStatusTablet state changed. Reason: New leader af6626c4d13c4eacbc3b536ba1da0fb7. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.764964 9718 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.764947 9717 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.768455 9743 mvcc.cc:205] Tried to move back new op lower bound from 7317214792680386560 to 7317214792427188224. Current Snapshot: MvccSnapshot[applied={T|T < 7317214792680386560}]
I20260811 06:23:12.772362 9719 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.772719 9719 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:12.773005 9744 mvcc.cc:205] Tried to move back new op lower bound from 7317214792680386560 to 7317214792427188224. Current Snapshot: MvccSnapshot[applied={T|T < 7317214792680386560}]
I20260811 06:23:12.773622 9736 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.773869 9736 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P af6626c4d13c4eacbc3b536ba1da0fb7: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:23:12.777529 9717 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.777827 9717 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P 51edfb55467f48138c512d78a0081bdf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.780123 9718 tablet_replica.cc:439] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } } }
I20260811 06:23:12.780395 9718 tablet_replica.cc:442] T 2eae6503439c4261aa3477b3048bb9ff P fbadd65ea7b64de48e2f5fe071e8183e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:23:12.788479 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.782097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.7.4.130:34007)
0811 06:23:12.782267 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.782273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.782477 (+ 204us) server_negotiation.cc:407] Connection header received
0811 06:23:12.782673 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.782679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.782804 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.782959 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:12.783405 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.784614 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.786001 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.786683 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.786813 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.787168 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.787843 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.787982 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.788077 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.788161 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:23:12.804302 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.798243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.7.4.130:55655)
0811 06:23:12.799000 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.799016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.799127 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.799502 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.799506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.799515 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:12.799791 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.799798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.801276 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.801279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.802290 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.802300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.803236 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.803240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.803399 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.803448 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.803859 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.803867 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.803926 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.804019 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":417,"threads_started":1}
I20260811 06:23:12.804345 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.798303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:55655 (local address 127.7.4.129:38279)
0811 06:23:12.798511 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.798517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.799098 (+ 581us) server_negotiation.cc:407] Connection header received
0811 06:23:12.799245 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.799250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.799377 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.799486 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:12.799956 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.801154 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.802422 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.803075 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.803199 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.803646 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.803762 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.803851 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.803977 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.804061 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:23:12.813401 9641 raft_consensus.cc:1261] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:12.814004 9332 raft_consensus.cc:1261] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:12.814075 9717 consensus_queue.cc:1035] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:12.814692 9717 consensus_queue.cc:1035] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:12.821204 9742 mvcc.cc:205] Tried to move back new op lower bound from 7317214792957841408 to 7317214791398895616. Current Snapshot: MvccSnapshot[applied={T|T < 7317214792957841408}]
I20260811 06:23:12.848888 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.841519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:34349 (local address 127.7.4.130:34007)
0811 06:23:12.841697 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:12.841701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:12.842529 (+ 828us) server_negotiation.cc:407] Connection header received
0811 06:23:12.842746 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:12.842752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:12.842834 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:12.842931 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:12.843960 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.845059 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.847071 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.847539 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.847677 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.848126 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.848225 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.848325 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:12.848484 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 06:23:12.848576 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:23:12.849014 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:12.841555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34007 (local address 127.7.4.129:34349)
0811 06:23:12.842405 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:12.842427 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:12.842587 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:12.843261 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:12.843265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:12.843277 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:12.843761 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.843775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.845200 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.845204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.846669 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:12.846682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:12.847739 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:12.847746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:12.847887 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:12.847938 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:12.848328 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:12.848342 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 06:23:12.848461 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:23:12.848618 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":396,"threads_started":1}
I20260811 06:23:12.889087 9150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:12.889366 9150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:12.893425 9150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Sending DeleteTablet for 3 replicas of tablet ce4692e725bd46d898a04ae657b43d46
I20260811 06:23:12.894586 9303 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4692e725bd46d898a04ae657b43d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:12 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36530
I20260811 06:23:12.894742 9456 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4692e725bd46d898a04ae657b43d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:12 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43012
I20260811 06:23:12.895816 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4692e725bd46d898a04ae657b43d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:12 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260811 06:23:12.903810 9764 tablet_replica.cc:330] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: stopping tablet replica
I20260811 06:23:12.903108 9150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:23:12.904305 9765 tablet_replica.cc:330] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: stopping tablet replica
I20260811 06:23:12.904309 9764 raft_consensus.cc:2229] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 06:23:12.904366 9150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:12.904722 9764 raft_consensus.cc:2258] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:12.904912 9765 raft_consensus.cc:2229] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:12.905673 9765 raft_consensus.cc:2258] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:12.911111 9765 ts_tablet_manager.cc:1905] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:12.912025 9303 tablet_service.cc:1467] Processing CreateTablet for tablet 142053b2baf6424c95118d84271e6739 (DEFAULT_TABLE table=test-table [id=092e01004dc64dfc9e91cbe963971d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.912525 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142053b2baf6424c95118d84271e6739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.912983 9456 tablet_service.cc:1467] Processing CreateTablet for tablet 142053b2baf6424c95118d84271e6739 (DEFAULT_TABLE table=test-table [id=092e01004dc64dfc9e91cbe963971d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.913405 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142053b2baf6424c95118d84271e6739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.917343 9610 tablet_service.cc:1467] Processing CreateTablet for tablet 142053b2baf6424c95118d84271e6739 (DEFAULT_TABLE table=test-table [id=092e01004dc64dfc9e91cbe963971d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:12.917851 9610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 142053b2baf6424c95118d84271e6739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:12.920078 9713 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Bootstrap starting.
I20260811 06:23:12.921979 9713 tablet_bootstrap.cc:654] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.925285 9711 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Bootstrap starting.
I20260811 06:23:12.926950 9711 tablet_bootstrap.cc:654] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.928786 9765 ts_tablet_manager.cc:1918] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:12.929052 9765 log.cc:1196] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/ce4692e725bd46d898a04ae657b43d46
I20260811 06:23:12.929775 9765 ts_tablet_manager.cc:1939] T ce4692e725bd46d898a04ae657b43d46 P 51edfb55467f48138c512d78a0081bdf: Deleting consensus metadata
I20260811 06:23:12.931579 9764 ts_tablet_manager.cc:1905] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:12.931982 9139 catalog_manager.cc:4841] TS 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007): tablet ce4692e725bd46d898a04ae657b43d46 (table test-table [id=f5413e4949be4196a3ae0c7daebea35c]) successfully deleted
I20260811 06:23:12.933633 9711 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: No bootstrap required, opened a new log
I20260811 06:23:12.933863 9711 ts_tablet_manager.cc:1397] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260811 06:23:12.934732 9711 raft_consensus.cc:348] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.935072 9711 raft_consensus.cc:374] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.935217 9711 raft_consensus.cc:729] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.935609 9711 consensus_queue.cc:260] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.936156 9712 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Bootstrap starting.
I20260811 06:23:12.936590 9711 ts_tablet_manager.cc:1428] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 06:23:12.937755 9712 tablet_bootstrap.cc:654] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:12.939512 9764 ts_tablet_manager.cc:1918] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:12.939718 9764 log.cc:1196] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/ce4692e725bd46d898a04ae657b43d46
I20260811 06:23:12.940346 9764 ts_tablet_manager.cc:1939] T ce4692e725bd46d898a04ae657b43d46 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting consensus metadata
I20260811 06:23:12.942298 9137 catalog_manager.cc:4841] TS af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279): tablet ce4692e725bd46d898a04ae657b43d46 (table test-table [id=f5413e4949be4196a3ae0c7daebea35c]) successfully deleted
I20260811 06:23:12.947953 9713 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: No bootstrap required, opened a new log
I20260811 06:23:12.948125 9713 ts_tablet_manager.cc:1397] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.006s
I20260811 06:23:12.948997 9766 tablet_replica.cc:330] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: stopping tablet replica
I20260811 06:23:12.949121 9712 tablet_bootstrap.cc:492] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: No bootstrap required, opened a new log
I20260811 06:23:12.949028 9713 raft_consensus.cc:348] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.949283 9712 ts_tablet_manager.cc:1397] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.006s
I20260811 06:23:12.949285 9713 raft_consensus.cc:374] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.949370 9713 raft_consensus.cc:729] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.950037 9712 raft_consensus.cc:348] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.950289 9712 raft_consensus.cc:374] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:12.950367 9712 raft_consensus.cc:729] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Initialized, Role: FOLLOWER
I20260811 06:23:12.950299 9766 raft_consensus.cc:2229] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:12.949743 9713 consensus_queue.cc:260] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.950899 9766 raft_consensus.cc:2258] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:12.951103 9713 ts_tablet_manager.cc:1428] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:12.952966 9766 ts_tablet_manager.cc:1905] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:12.950659 9712 consensus_queue.cc:260] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:12.955214 9712 ts_tablet_manager.cc:1428] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260811 06:23:12.960984 9766 ts_tablet_manager.cc:1918] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:12.961158 9766 log.cc:1196] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/ce4692e725bd46d898a04ae657b43d46
I20260811 06:23:12.961709 9766 ts_tablet_manager.cc:1939] T ce4692e725bd46d898a04ae657b43d46 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting consensus metadata
I20260811 06:23:12.963629 9138 catalog_manager.cc:4841] TS fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905): tablet ce4692e725bd46d898a04ae657b43d46 (table test-table [id=f5413e4949be4196a3ae0c7daebea35c]) successfully deleted
I20260811 06:23:13.175570 9718 raft_consensus.cc:482] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:13.175870 9718 raft_consensus.cc:504] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:13.177950 9718 leader_election.cc:288] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007)
I20260811 06:23:13.190186 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.179447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:46457 (local address 127.7.4.129:38279)
0811 06:23:13.179633 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:13.179639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:13.184062 (+ 4423us) server_negotiation.cc:407] Connection header received
0811 06:23:13.184385 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:13.184391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:13.184483 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:13.184600 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:13.185146 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.186261 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.188285 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.189185 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.189313 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.189500 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:13.189741 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 06:23:13.189879 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:23:13.190306 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.178210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.7.4.131:46457)
0811 06:23:13.183915 (+ 5705us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:13.183940 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:13.184189 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:13.184624 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:13.184629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:13.184654 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:13.184950 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.184962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.186489 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.186495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.188108 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.188121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.189371 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.189377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.189555 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.189590 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:23:13.189698 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:23:13.189902 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5577,"thread_start_us":5457,"threads_started":1}
I20260811 06:23:13.191102 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142053b2baf6424c95118d84271e6739" candidate_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" is_pre_election: true
I20260811 06:23:13.191426 9332 raft_consensus.cc:2454] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbadd65ea7b64de48e2f5fe071e8183e in term 0.
I20260811 06:23:13.191761 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.178577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34007 (local address 127.7.4.131:55087)
0811 06:23:13.184259 (+ 5682us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:13.184279 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:13.184396 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:13.184931 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:13.184935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:13.184956 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:13.185312 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.185320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.186503 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.186509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.188822 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.188838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.190838 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.190844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.191044 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.191080 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:23:13.191195 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:23:13.191390 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5567,"thread_start_us":5403,"threads_started":1}
I20260811 06:23:13.192060 9560 leader_election.cc:302] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af6626c4d13c4eacbc3b536ba1da0fb7, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:13.192364 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.179027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:55087 (local address 127.7.4.130:34007)
0811 06:23:13.179244 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:13.179251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:13.184366 (+ 5115us) server_negotiation.cc:407] Connection header received
0811 06:23:13.184577 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:13.184583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:13.184670 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:13.184765 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:13.185482 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.186312 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.189007 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.190641 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.190779 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.190880 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:13.191243 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 06:23:13.192069 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:23:13.192548 9718 raft_consensus.cc:2781] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:13.192687 9718 raft_consensus.cc:482] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:13.192804 9718 raft_consensus.cc:3037] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:13.192911 9478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142053b2baf6424c95118d84271e6739" candidate_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51edfb55467f48138c512d78a0081bdf" is_pre_election: true
I20260811 06:23:13.193153 9478 raft_consensus.cc:2454] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbadd65ea7b64de48e2f5fe071e8183e in term 0.
I20260811 06:23:13.196336 9718 raft_consensus.cc:504] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:13.197069 9718 leader_election.cc:288] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [CANDIDATE]: Term 1 election: Requested vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007)
I20260811 06:23:13.197573 9478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142053b2baf6424c95118d84271e6739" candidate_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51edfb55467f48138c512d78a0081bdf"
I20260811 06:23:13.197633 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142053b2baf6424c95118d84271e6739" candidate_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
I20260811 06:23:13.197820 9478 raft_consensus.cc:3037] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:13.197926 9332 raft_consensus.cc:3037] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:13.201126 9332 raft_consensus.cc:2454] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbadd65ea7b64de48e2f5fe071e8183e in term 1.
I20260811 06:23:13.201175 9478 raft_consensus.cc:2454] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbadd65ea7b64de48e2f5fe071e8183e in term 1.
I20260811 06:23:13.201642 9562 leader_election.cc:302] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:13.202039 9718 raft_consensus.cc:2781] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:13.202646 9718 raft_consensus.cc:686] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 LEADER]: Becoming Leader. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Running, Role: LEADER
I20260811 06:23:13.203105 9718 consensus_queue.cc:237] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:13.207895 9150 catalog_manager.cc:5495] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e reported cstate change: term changed from 0 to 1, leader changed from <none> to fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131). New cstate: current_term: 1 leader_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } health_report { overall_health: HEALTHY } } }
I20260811 06:23:13.251977 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.238603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.7.4.131:36905)
0811 06:23:13.245437 (+ 6834us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:13.245444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:13.245468 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:23:13.245582 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:13.245587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:13.245705 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:13.245854 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:13.246375 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.247164 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.249166 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.249714 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.249815 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.250438 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:13.251154 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:13.251270 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:13.251530 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 06:23:13.251596 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6685,"thread_start_us":6525,"threads_started":1}
I20260811 06:23:13.269450 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.263475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:59473 (local address 127.7.4.129:38279)
0811 06:23:13.263644 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:13.263649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:13.264343 (+ 694us) server_negotiation.cc:407] Connection header received
0811 06:23:13.264592 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:13.264599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:13.264673 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:13.264773 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:13.265405 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.266146 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.267653 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.268117 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.268262 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.268717 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:13.268825 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:13.268923 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:13.269061 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 06:23:13.269146 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:23:13.269661 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.263483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38279 (local address 127.7.4.131:59473)
0811 06:23:13.264204 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:13.264219 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:13.264402 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:13.264810 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:13.264815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:13.264828 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:13.265255 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.265265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.266285 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.266290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.267503 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.267515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.268301 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.268310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.268463 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.268513 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:13.268926 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:13.268940 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 06:23:13.269038 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:23:13.269168 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":410,"threads_started":1}
I20260811 06:23:13.278694 9478 raft_consensus.cc:1261] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Refusing update from remote peer fbadd65ea7b64de48e2f5fe071e8183e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:13.278757 9332 raft_consensus.cc:1261] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Refusing update from remote peer fbadd65ea7b64de48e2f5fe071e8183e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:13.279495 9772 consensus_queue.cc:1035] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:13.280139 9718 consensus_queue.cc:1035] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:13.313055 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.303652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:59713 (local address 127.7.4.131:36905)
0811 06:23:13.303974 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:13.303980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:13.303999 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:23:13.304073 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:13.304078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:13.304162 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:13.304255 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:13.305223 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.306423 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.309701 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.310721 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.310879 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.311761 (+ 882us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:13.311934 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:13.312028 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:13.312496 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 06:23:13.312612 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260811 06:23:13.313987 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:13.302819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:36905 (local address 127.7.4.129:59713)
0811 06:23:13.303188 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:13.303224 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:13.303387 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:13.304779 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:13.304784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:13.304816 (+ 32us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:23:13.305019 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.305031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.306612 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.306621 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.308935 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:13.308956 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:13.311046 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:13.311056 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:13.311461 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:13.311536 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:23:13.312097 (+ 561us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:23:13.312112 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 06:23:13.312205 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 06:23:13.313444 (+ 1239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
W20260811 06:23:13.315627 9744 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:23:14.908061 9150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:14.908351 9150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:14.912663 9150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Sending DeleteTablet for 3 replicas of tablet 142053b2baf6424c95118d84271e6739
I20260811 06:23:14.913411 9303 tablet_service.cc:1514] Processing DeleteTablet for tablet 142053b2baf6424c95118d84271e6739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:14 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36530
I20260811 06:23:14.913635 9456 tablet_service.cc:1514] Processing DeleteTablet for tablet 142053b2baf6424c95118d84271e6739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:14 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43012
I20260811 06:23:14.913945 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 142053b2baf6424c95118d84271e6739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:14 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260811 06:23:14.923831 9808 tablet_replica.cc:330] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: stopping tablet replica
I20260811 06:23:14.924117 9808 raft_consensus.cc:2229] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:14.924253 9810 tablet_replica.cc:330] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: stopping tablet replica
I20260811 06:23:14.924431 9808 raft_consensus.cc:2258] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:14.924641 9810 raft_consensus.cc:2229] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:14.925009 9810 raft_consensus.cc:2258] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:14.925094 9811 tablet_replica.cc:330] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: stopping tablet replica
I20260811 06:23:14.925321 9811 raft_consensus.cc:2229] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:14.925915 9811 raft_consensus.cc:2258] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:14.925962 9810 ts_tablet_manager.cc:1905] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:14.926803 9808 ts_tablet_manager.cc:1905] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:14.927207 9811 ts_tablet_manager.cc:1905] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:14.932545 9808 ts_tablet_manager.cc:1918] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:14.932695 9808 log.cc:1196] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/142053b2baf6424c95118d84271e6739
I20260811 06:23:14.932947 9810 ts_tablet_manager.cc:1918] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:14.933094 9810 log.cc:1196] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/142053b2baf6424c95118d84271e6739
I20260811 06:23:14.933247 9808 ts_tablet_manager.cc:1939] T 142053b2baf6424c95118d84271e6739 P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting consensus metadata
I20260811 06:23:14.933624 9810 ts_tablet_manager.cc:1939] T 142053b2baf6424c95118d84271e6739 P 51edfb55467f48138c512d78a0081bdf: Deleting consensus metadata
I20260811 06:23:14.934334 9137 catalog_manager.cc:4841] TS af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279): tablet 142053b2baf6424c95118d84271e6739 (table test-table [id=092e01004dc64dfc9e91cbe963971d17]) successfully deleted
I20260811 06:23:14.934428 9811 ts_tablet_manager.cc:1918] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:14.934540 9811 log.cc:1196] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/142053b2baf6424c95118d84271e6739
I20260811 06:23:14.935040 9811 ts_tablet_manager.cc:1939] T 142053b2baf6424c95118d84271e6739 P fbadd65ea7b64de48e2f5fe071e8183e: Deleting consensus metadata
I20260811 06:23:14.935117 9139 catalog_manager.cc:4841] TS 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007): tablet 142053b2baf6424c95118d84271e6739 (table test-table [id=092e01004dc64dfc9e91cbe963971d17]) successfully deleted
I20260811 06:23:14.936074 9138 catalog_manager.cc:4841] TS fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905): tablet 142053b2baf6424c95118d84271e6739 (table test-table [id=092e01004dc64dfc9e91cbe963971d17]) successfully deleted
Aug 11 06:23:14 dist-test-slave-07c2 krb5kdc[9106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:23:14 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:14.979519 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:14.965478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.7.4.190:45515)
0811 06:23:14.966233 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:14.966237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:14.966261 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:23:14.966357 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:14.966360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:14.966420 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:14.966514 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:14.967752 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:14.968601 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:14.969837 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:14.970246 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:14.970390 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:14.974243 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:14.974248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:14.974251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:14.974281 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:14.977081 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:14.977661 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:14.977667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:14.977670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:14.977811 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:14.978145 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:14.978152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:14.978156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:14.978402 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:14.978638 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:14.978922 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:23:14.979123 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":405,"threads_started":1}
I20260811 06:23:14.982810 9150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:14.983493 9150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:14.989758 9303 tablet_service.cc:1467] Processing CreateTablet for tablet 8857c6369b31497b8a190291cae0b5af (DEFAULT_TABLE table=test-table [id=b3e74bb603184f8f8742dd17cfb36c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:14.989876 9456 tablet_service.cc:1467] Processing CreateTablet for tablet 8857c6369b31497b8a190291cae0b5af (DEFAULT_TABLE table=test-table [id=b3e74bb603184f8f8742dd17cfb36c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:14.990237 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8857c6369b31497b8a190291cae0b5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:14.990334 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8857c6369b31497b8a190291cae0b5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:14.990340 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 8857c6369b31497b8a190291cae0b5af (DEFAULT_TABLE table=test-table [id=b3e74bb603184f8f8742dd17cfb36c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:14.990772 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8857c6369b31497b8a190291cae0b5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:14.995412 9820 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Bootstrap starting.
I20260811 06:23:14.996940 9820 tablet_bootstrap.cc:654] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:14.997869 9821 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Bootstrap starting.
I20260811 06:23:14.998445 9822 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Bootstrap starting.
I20260811 06:23:14.999737 9821 tablet_bootstrap.cc:654] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:14.999992 9820 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: No bootstrap required, opened a new log
I20260811 06:23:15.000149 9820 ts_tablet_manager.cc:1397] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:23:15.000254 9822 tablet_bootstrap.cc:654] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:15.000969 9820 raft_consensus.cc:348] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.001246 9820 raft_consensus.cc:374] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.001377 9820 raft_consensus.cc:729] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.001649 9820 consensus_queue.cc:260] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.001914 9821 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: No bootstrap required, opened a new log
I20260811 06:23:15.002058 9821 ts_tablet_manager.cc:1397] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 06:23:15.002175 9820 ts_tablet_manager.cc:1428] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:23:15.002877 9822 tablet_bootstrap.cc:492] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: No bootstrap required, opened a new log
I20260811 06:23:15.002861 9821 raft_consensus.cc:348] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.003021 9822 ts_tablet_manager.cc:1397] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 06:23:15.003105 9821 raft_consensus.cc:374] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.003204 9821 raft_consensus.cc:729] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.003502 9821 consensus_queue.cc:260] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.003845 9822 raft_consensus.cc:348] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.004117 9822 raft_consensus.cc:374] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.004231 9822 raft_consensus.cc:729] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.004487 9822 consensus_queue.cc:260] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.005039 9822 ts_tablet_manager.cc:1428] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 06:23:15.006743 9821 ts_tablet_manager.cc:1428] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 06:23:15.148424 9827 raft_consensus.cc:482] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:15.148710 9827 raft_consensus.cc:504] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.149411 9827 leader_election.cc:288] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:15.150092 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8857c6369b31497b8a190291cae0b5af" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" is_pre_election: true
I20260811 06:23:15.150092 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8857c6369b31497b8a190291cae0b5af" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" is_pre_election: true
I20260811 06:23:15.150346 9332 raft_consensus.cc:2454] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:15.150346 9641 raft_consensus.cc:2454] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:15.150849 9408 leader_election.cc:302] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, af6626c4d13c4eacbc3b536ba1da0fb7, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:15.151142 9827 raft_consensus.cc:2781] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:15.151270 9827 raft_consensus.cc:482] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:15.151419 9827 raft_consensus.cc:3037] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.153471 9827 raft_consensus.cc:504] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.153996 9827 leader_election.cc:288] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Requested vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:15.154524 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8857c6369b31497b8a190291cae0b5af" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
I20260811 06:23:15.154560 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8857c6369b31497b8a190291cae0b5af" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
I20260811 06:23:15.154759 9332 raft_consensus.cc:3037] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.154757 9641 raft_consensus.cc:3037] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.157887 9641 raft_consensus.cc:2454] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:15.157893 9332 raft_consensus.cc:2454] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:15.158294 9405 leader_election.cc:302] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, af6626c4d13c4eacbc3b536ba1da0fb7; no voters:
I20260811 06:23:15.158619 9827 raft_consensus.cc:2781] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:15.159091 9827 raft_consensus.cc:686] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Becoming Leader. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Running, Role: LEADER
I20260811 06:23:15.159688 9827 consensus_queue.cc:237] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.162621 9152 catalog_manager.cc:5495] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 51edfb55467f48138c512d78a0081bdf (127.7.4.130). New cstate: current_term: 1 leader_uuid: "51edfb55467f48138c512d78a0081bdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20260811 06:23:15.201020 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:15.195233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.7.4.130:34007)
0811 06:23:15.195860 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:15.195864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:15.195881 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:23:15.195927 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:15.195931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:15.195988 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:15.196096 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:15.196704 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:15.197577 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:15.198906 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:15.199593 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:15.199715 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:15.200178 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:15.200311 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:15.200392 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:15.200529 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:23:15.200604 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":325,"threads_started":1}
I20260811 06:23:15.210114 9641 raft_consensus.cc:1261] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:15.210170 9332 raft_consensus.cc:1261] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:15.210759 9828 consensus_queue.cc:1035] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:15.211154 9827 consensus_queue.cc:1035] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:15.266033 9152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:15.266256 9152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:15.269263 9152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Sending DeleteTablet for 3 replicas of tablet 8857c6369b31497b8a190291cae0b5af
I20260811 06:23:15.270169 9303 tablet_service.cc:1514] Processing DeleteTablet for tablet 8857c6369b31497b8a190291cae0b5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36530
I20260811 06:23:15.270280 9456 tablet_service.cc:1514] Processing DeleteTablet for tablet 8857c6369b31497b8a190291cae0b5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43012
I20260811 06:23:15.270421 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 8857c6369b31497b8a190291cae0b5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260811 06:23:15.270506 9808 tablet_replica.cc:330] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: stopping tablet replica
I20260811 06:23:15.270568 9810 tablet_replica.cc:330] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: stopping tablet replica
I20260811 06:23:15.270720 9811 tablet_replica.cc:330] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: stopping tablet replica
I20260811 06:23:15.270712 9808 raft_consensus.cc:2229] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:15.270773 9810 raft_consensus.cc:2229] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:15.270921 9808 raft_consensus.cc:2258] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:15.270920 9811 raft_consensus.cc:2229] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:15.271108 9811 raft_consensus.cc:2258] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:15.271142 9810 raft_consensus.cc:2258] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:15.272377 9811 ts_tablet_manager.cc:1905] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:15.272186 9152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:23:15.272377 9808 ts_tablet_manager.cc:1905] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:15.272624 9810 ts_tablet_manager.cc:1905] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 06:23:15.272876 9152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:15.278625 9810 ts_tablet_manager.cc:1918] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:15.278864 9810 log.cc:1196] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/8857c6369b31497b8a190291cae0b5af
I20260811 06:23:15.279276 9810 ts_tablet_manager.cc:1939] T 8857c6369b31497b8a190291cae0b5af P 51edfb55467f48138c512d78a0081bdf: Deleting consensus metadata
I20260811 06:23:15.279865 9808 ts_tablet_manager.cc:1918] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:15.279865 9811 ts_tablet_manager.cc:1918] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:23:15.280035 9808 log.cc:1196] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/8857c6369b31497b8a190291cae0b5af
I20260811 06:23:15.280176 9811 log.cc:1196] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/8857c6369b31497b8a190291cae0b5af
I20260811 06:23:15.280495 9139 catalog_manager.cc:4841] TS 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007): tablet 8857c6369b31497b8a190291cae0b5af (table test-table [id=b3e74bb603184f8f8742dd17cfb36c57]) successfully deleted
I20260811 06:23:15.280592 9808 ts_tablet_manager.cc:1939] T 8857c6369b31497b8a190291cae0b5af P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting consensus metadata
I20260811 06:23:15.281340 9811 ts_tablet_manager.cc:1939] T 8857c6369b31497b8a190291cae0b5af P fbadd65ea7b64de48e2f5fe071e8183e: Deleting consensus metadata
I20260811 06:23:15.281737 9137 catalog_manager.cc:4841] TS af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279): tablet 8857c6369b31497b8a190291cae0b5af (table test-table [id=b3e74bb603184f8f8742dd17cfb36c57]) successfully deleted
I20260811 06:23:15.282903 9138 catalog_manager.cc:4841] TS fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905): tablet 8857c6369b31497b8a190291cae0b5af (table test-table [id=b3e74bb603184f8f8742dd17cfb36c57]) successfully deleted
I20260811 06:23:15.284740 9303 tablet_service.cc:1467] Processing CreateTablet for tablet f23e82263d444827bf34acdeed90d26d (DEFAULT_TABLE table=test-table [id=ea477620f0da4451a35b4fbc6234b4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:15.285117 9456 tablet_service.cc:1467] Processing CreateTablet for tablet f23e82263d444827bf34acdeed90d26d (DEFAULT_TABLE table=test-table [id=ea477620f0da4451a35b4fbc6234b4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:15.285226 9303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23e82263d444827bf34acdeed90d26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:15.285526 9456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23e82263d444827bf34acdeed90d26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:15.290805 9611 tablet_service.cc:1467] Processing CreateTablet for tablet f23e82263d444827bf34acdeed90d26d (DEFAULT_TABLE table=test-table [id=ea477620f0da4451a35b4fbc6234b4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:15.290993 9821 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Bootstrap starting.
I20260811 06:23:15.291371 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f23e82263d444827bf34acdeed90d26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:15.291826 9820 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Bootstrap starting.
I20260811 06:23:15.293088 9821 tablet_bootstrap.cc:654] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:15.293394 9820 tablet_bootstrap.cc:654] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:15.297425 9821 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: No bootstrap required, opened a new log
I20260811 06:23:15.297488 9820 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: No bootstrap required, opened a new log
I20260811 06:23:15.297653 9821 ts_tablet_manager.cc:1397] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:23:15.297708 9820 ts_tablet_manager.cc:1397] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 06:23:15.298388 9821 raft_consensus.cc:348] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.298638 9821 raft_consensus.cc:374] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.298592 9820 raft_consensus.cc:348] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.298853 9820 raft_consensus.cc:374] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.298944 9820 raft_consensus.cc:729] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6626c4d13c4eacbc3b536ba1da0fb7, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.299064 9821 raft_consensus.cc:729] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.299556 9822 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Bootstrap starting.
I20260811 06:23:15.299401 9821 consensus_queue.cc:260] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.299445 9820 consensus_queue.cc:260] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.300040 9821 ts_tablet_manager.cc:1428] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:15.300961 9820 ts_tablet_manager.cc:1428] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:15.301491 9822 tablet_bootstrap.cc:654] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:15.312286 9822 tablet_bootstrap.cc:492] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: No bootstrap required, opened a new log
I20260811 06:23:15.312453 9822 ts_tablet_manager.cc:1397] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260811 06:23:15.313090 9822 raft_consensus.cc:348] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.313354 9822 raft_consensus.cc:374] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:15.313427 9822 raft_consensus.cc:729] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbadd65ea7b64de48e2f5fe071e8183e, State: Initialized, Role: FOLLOWER
I20260811 06:23:15.313628 9822 consensus_queue.cc:260] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.314160 9822 ts_tablet_manager.cc:1428] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:23:15.356086 9828 raft_consensus.cc:482] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:15.356369 9828 raft_consensus.cc:504] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.357168 9828 leader_election.cc:288] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:15.357925 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23e82263d444827bf34acdeed90d26d" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" is_pre_election: true
I20260811 06:23:15.357932 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23e82263d444827bf34acdeed90d26d" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" is_pre_election: true
I20260811 06:23:15.358284 9332 raft_consensus.cc:2454] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:15.358310 9641 raft_consensus.cc:2454] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 0.
I20260811 06:23:15.358871 9408 leader_election.cc:302] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:15.359241 9828 raft_consensus.cc:2781] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:15.359414 9828 raft_consensus.cc:482] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:15.359555 9828 raft_consensus.cc:3037] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.361780 9828 raft_consensus.cc:504] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.362576 9828 leader_election.cc:288] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Requested vote from peers af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279), fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905)
I20260811 06:23:15.363344 9332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23e82263d444827bf34acdeed90d26d" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7"
I20260811 06:23:15.363381 9641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f23e82263d444827bf34acdeed90d26d" candidate_uuid: "51edfb55467f48138c512d78a0081bdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbadd65ea7b64de48e2f5fe071e8183e"
I20260811 06:23:15.363693 9332 raft_consensus.cc:3037] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.363641 9641 raft_consensus.cc:3037] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:15.367347 9641 raft_consensus.cc:2454] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:15.367328 9332 raft_consensus.cc:2454] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51edfb55467f48138c512d78a0081bdf in term 1.
I20260811 06:23:15.368032 9408 leader_election.cc:302] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51edfb55467f48138c512d78a0081bdf, fbadd65ea7b64de48e2f5fe071e8183e; no voters:
I20260811 06:23:15.368448 9828 raft_consensus.cc:2781] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:15.368734 9828 raft_consensus.cc:686] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Becoming Leader. State: Replica: 51edfb55467f48138c512d78a0081bdf, State: Running, Role: LEADER
I20260811 06:23:15.369060 9828 consensus_queue.cc:237] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } }
I20260811 06:23:15.372639 9152 catalog_manager.cc:5495] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 51edfb55467f48138c512d78a0081bdf (127.7.4.130). New cstate: current_term: 1 leader_uuid: "51edfb55467f48138c512d78a0081bdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51edfb55467f48138c512d78a0081bdf" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20260811 06:23:15.411705 9332 raft_consensus.cc:1261] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:15.411702 9641 raft_consensus.cc:1261] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Refusing update from remote peer 51edfb55467f48138c512d78a0081bdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:15.412431 9828 consensus_queue.cc:1035] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbadd65ea7b64de48e2f5fe071e8183e" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:15.412904 9836 consensus_queue.cc:1035] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "af6626c4d13c4eacbc3b536ba1da0fb7" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:23:15.435470 9788 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:23:17.020732 9151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:17.020943 9151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:17.025029 9151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e104e610c0e45fb8d14f77524763748: Sending DeleteTablet for 3 replicas of tablet f23e82263d444827bf34acdeed90d26d
I20260811 06:23:17.025799 9303 tablet_service.cc:1514] Processing DeleteTablet for tablet f23e82263d444827bf34acdeed90d26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:17 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36530
I20260811 06:23:17.025952 9456 tablet_service.cc:1514] Processing DeleteTablet for tablet f23e82263d444827bf34acdeed90d26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:17 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43012
I20260811 06:23:17.026011 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet f23e82263d444827bf34acdeed90d26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:17 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35930
I20260811 06:23:17.026559 9866 tablet_replica.cc:330] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: stopping tablet replica
I20260811 06:23:17.026697 9867 tablet_replica.cc:330] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: stopping tablet replica
I20260811 06:23:17.026823 9866 raft_consensus.cc:2229] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:17.026959 9868 tablet_replica.cc:330] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: stopping tablet replica
I20260811 06:23:17.027030 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9232
I20260811 06:23:17.026960 9867 raft_consensus.cc:2229] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:17.027215 9866 raft_consensus.cc:2258] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:17.027211 9868 raft_consensus.cc:2229] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:17.027539 9867 raft_consensus.cc:2258] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:17.027592 9868 raft_consensus.cc:2258] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:17.028717 9867 ts_tablet_manager.cc:1905] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:17.029356 9866 ts_tablet_manager.cc:1905] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:17.029465 9868 ts_tablet_manager.cc:1905] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:17.036180 9866 ts_tablet_manager.cc:1918] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:17.036180 9868 ts_tablet_manager.cc:1918] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:17.036334 9868 log.cc:1196] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/f23e82263d444827bf34acdeed90d26d
I20260811 06:23:17.036877 9866 log.cc:1196] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/f23e82263d444827bf34acdeed90d26d
I20260811 06:23:17.037043 9867 ts_tablet_manager.cc:1918] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:23:17.037272 9867 log.cc:1196] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/f23e82263d444827bf34acdeed90d26d
I20260811 06:23:17.037537 9868 ts_tablet_manager.cc:1939] T f23e82263d444827bf34acdeed90d26d P fbadd65ea7b64de48e2f5fe071e8183e: Deleting consensus metadata
I20260811 06:23:17.037545 9866 ts_tablet_manager.cc:1939] T f23e82263d444827bf34acdeed90d26d P af6626c4d13c4eacbc3b536ba1da0fb7: Deleting consensus metadata
I20260811 06:23:17.037802 9867 ts_tablet_manager.cc:1939] T f23e82263d444827bf34acdeed90d26d P 51edfb55467f48138c512d78a0081bdf: Deleting consensus metadata
I20260811 06:23:17.038683 9137 catalog_manager.cc:4841] TS af6626c4d13c4eacbc3b536ba1da0fb7 (127.7.4.129:38279): tablet f23e82263d444827bf34acdeed90d26d (table test-table [id=ea477620f0da4451a35b4fbc6234b4ae]) successfully deleted
I20260811 06:23:17.038980 9139 catalog_manager.cc:4841] TS 51edfb55467f48138c512d78a0081bdf (127.7.4.130:34007): tablet f23e82263d444827bf34acdeed90d26d (table test-table [id=ea477620f0da4451a35b4fbc6234b4ae]) successfully deleted
I20260811 06:23:17.039181 9138 catalog_manager.cc:4841] TS fbadd65ea7b64de48e2f5fe071e8183e (127.7.4.131:36905): tablet f23e82263d444827bf34acdeed90d26d (table test-table [id=ea477620f0da4451a35b4fbc6234b4ae]) successfully deleted
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:17.057093 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:17.029819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.7.4.129:38279)
0811 06:23:17.036850 (+ 7031us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:17.036855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:17.036880 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:23:17.037066 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:17.037073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:17.037191 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:17.037283 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:17.039315 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.040109 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.041356 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.041729 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.041817 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:17.051219 (+ 9402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:17.051235 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:17.051252 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:17.051344 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:17.054328 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.054998 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.055005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.055007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.055143 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.055571 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.055576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.055578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.055974 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:17.056244 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:17.056444 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 06:23:17.056661 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6874,"spinlock_wait_cycles":10112,"thread_start_us":6856,"threads_started":1}
W20260811 06:23:17.057827 9376 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36542
W20260811 06:23:17.058519 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9232: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:17.058692 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9232
I20260811 06:23:17.086853 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9385
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:17.102478 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:17.087703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.7.4.130:34007)
0811 06:23:17.088201 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:17.088206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:17.088229 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:23:17.088284 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:17.088288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:17.088353 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:17.088444 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:17.089743 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.090548 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.092038 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.092462 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.092596 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:17.096755 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:17.096769 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:17.096786 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:17.096819 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:17.099942 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.100648 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.100655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.100658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.100785 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.101099 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.101102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.101104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.101447 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:17.101679 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:17.101889 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 06:23:17.102057 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
W20260811 06:23:17.103173 9529 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43040
W20260811 06:23:17.103827 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9385: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:17.104007 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9385
I20260811 06:23:17.132211 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9541
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429394, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:17.147804 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:17.133018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.7.4.131:36905)
0811 06:23:17.133620 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:17.133624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:17.133643 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:23:17.133698 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:17.133701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:17.133761 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:17.133852 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:17.135186 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.135964 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.137135 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:17.137509 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:17.137604 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:17.141968 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:17.141983 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:17.142000 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:17.142038 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:17.145123 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.145825 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.145832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.145834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.145955 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:17.146300 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:17.146303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:17.146305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:17.146634 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:17.146871 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:17.147184 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:23:17.147462 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":345,"threads_started":1}
W20260811 06:23:17.148487 9685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35958
W20260811 06:23:17.149194 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9541: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:17.149355 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9541
I20260811 06:23:17.179075 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9119
I20260811 06:23:17.180413 9197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:17.303324 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9119
2026-08-11T06:23:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8587 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9877](info): set up 2 sockets
Aug 11 06:23:17 dist-test-slave-07c2 krb5kdc[9877](info): commencing operation
krb5kdc: starting...
W20260811 06:23:18.545676 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:18 dist-test-slave-07c2 krb5kdc[9877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:18.767892 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36249
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:36249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:19.096020 9892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:19.096323 9892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:19.096478 9892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:19.103856 9892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:19.103965 9892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:19.104069 9892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:19.104161 9892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:19.104212 9892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:19.114182 9892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:36249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:19.115600 9892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:19.116945 9892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:19.129032 9898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:19.129130 9900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:19.129097 9897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:19.129732 9892 server_base.cc:1034] running on GCE node
I20260811 06:23:19.130607 9892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:19.132081 9892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:19.133289 9892 hybrid_clock.cc:648] HybridClock initialized: now 1786429399133250 us; error 43 us; skew 500 ppm
I20260811 06:23:19.133649 9892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:19 dist-test-slave-07c2 krb5kdc[9877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429399, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:19.136906 9892 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:19.138487 9892 webserver.cc:460] Webserver started at http://127.7.4.190:35235/ using document root <none> and password file <none>
I20260811 06:23:19.139190 9892 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:19.139365 9892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:19.139777 9892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:19.142858 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "94b3d7205a8e4d73b4e7dbab07ed5632"
server_key_iv: "bcf0d4dcc03c571a9914bb7615bb9521"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.143671 9892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "94b3d7205a8e4d73b4e7dbab07ed5632"
server_key_iv: "bcf0d4dcc03c571a9914bb7615bb9521"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.148698 9892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:23:19.152172 9908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:19.153462 9892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:19.153646 9892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "94b3d7205a8e4d73b4e7dbab07ed5632"
server_key_iv: "bcf0d4dcc03c571a9914bb7615bb9521"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.153846 9892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:19.168244 9892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:19.172259 9892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:19.172540 9892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:19.191826 9892 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:36249
I20260811 06:23:19.191859 9962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:36249 every 8 connection(s)
I20260811 06:23:19.193295 9892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:19.197131 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:19.197358 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9892
I20260811 06:23:19.197593 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:19.198241 7186 external_mini_cluster.cc:1428] Setting key be99fd0a70a467599ecdf1812dc77c18
I20260811 06:23:19.212000 9963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467: Bootstrap starting.
Aug 11 06:23:19 dist-test-slave-07c2 krb5kdc[9877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:19.215278 9963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:19.216693 9963 log.cc:824] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:19.220073 9963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467: No bootstrap required, opened a new log
I20260811 06:23:19.223001 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:19.200261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.7.4.190:36249)
0811 06:23:19.201150 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:19.201173 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:19.201227 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:23:19.203986 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:19.204017 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:19.204560 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:19.204997 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:19.206391 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.208459 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.209714 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.210760 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.210947 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:19.214908 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:19.214919 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:19.214939 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:19.214984 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:19.218879 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:19.219628 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:19.219636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:19.219644 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:19.219855 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:19.220249 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:19.220253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:19.220255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:19.220791 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:19.221094 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:19.221570 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 06:23:19.222002 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":341,"threads_started":1}
I20260811 06:23:19.228019 9963 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } }
I20260811 06:23:19.228390 9963 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:19.228497 9963 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9bfdda70c4d4ed2a3212fea8bbb7467, State: Initialized, Role: FOLLOWER
I20260811 06:23:19.229125 9963 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } }
I20260811 06:23:19.229399 9963 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:19.229564 9963 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:19.229755 9963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:19.232113 9963 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } }
I20260811 06:23:19.232602 9963 leader_election.cc:302] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9bfdda70c4d4ed2a3212fea8bbb7467; no voters:
I20260811 06:23:19.233194 9963 leader_election.cc:288] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:19.233335 9968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:19.233975 9968 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [term 1 LEADER]: Becoming Leader. State: Replica: d9bfdda70c4d4ed2a3212fea8bbb7467, State: Running, Role: LEADER
I20260811 06:23:19.234504 9968 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } }
I20260811 06:23:19.235221 9963 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:19.237093 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } } }
I20260811 06:23:19.237475 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:19.238446 9974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:19.238247 9970 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9bfdda70c4d4ed2a3212fea8bbb7467. Latest consensus state: current_term: 1 leader_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9bfdda70c4d4ed2a3212fea8bbb7467" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36249 } } }
I20260811 06:23:19.238559 9970 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:19.245577 9974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:19.254644 9974 catalog_manager.cc:1347] Generated new cluster ID: 707e2580fb3048c8b0c7b21e855bc182
I20260811 06:23:19.254755 9974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:19.305009 9974 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:19.306845 9974 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:19.324339 9974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9bfdda70c4d4ed2a3212fea8bbb7467: Generated new TSK 0
I20260811 06:23:19.325150 9974 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:19.389189 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:19.738896 9990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:19.739280 9990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:19.739460 9990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:19.747206 9990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:19.747413 9990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:19.747632 9990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:19.758178 9990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:19.759685 9990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:19.761086 9990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:19.774911 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:19.774886 9998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:19.774957 9996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:19.775926 9990 server_base.cc:1034] running on GCE node
I20260811 06:23:19.776651 9990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:19.777730 9990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:19.779000 9990 hybrid_clock.cc:648] HybridClock initialized: now 1786429399778944 us; error 46 us; skew 500 ppm
I20260811 06:23:19.779444 9990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:19 dist-test-slave-07c2 krb5kdc[9877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429399, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:19.782734 9990 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:19.784317 9990 webserver.cc:460] Webserver started at http://127.7.4.129:35443/ using document root <none> and password file <none>
I20260811 06:23:19.785032 9990 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:19.785185 9990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:19.785530 9990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:19.788782 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "c2e5917561bf43679783331a0ebfef32"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "a2256eb8bd76ec980cd45bb3c04856d4"
server_key_iv: "6d4130569f0f13430a3890bb92a8c9dc"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.789613 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "c2e5917561bf43679783331a0ebfef32"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "a2256eb8bd76ec980cd45bb3c04856d4"
server_key_iv: "6d4130569f0f13430a3890bb92a8c9dc"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.794945 9990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:19.798501 10007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:19.799872 9990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:19.800050 9990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "c2e5917561bf43679783331a0ebfef32"
format_stamp: "Formatted at 2026-08-11 06:23:19 on dist-test-slave-07c2"
server_key: "a2256eb8bd76ec980cd45bb3c04856d4"
server_key_iv: "6d4130569f0f13430a3890bb92a8c9dc"
server_key_version: "encryptionkey@0"
I20260811 06:23:19.800269 9990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:19.823657 9990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:19.828073 9990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:19.828393 9990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:19.829396 9990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:19.830747 9990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:19.830850 9990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:19.830993 9990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:19.831063 9990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:19.865360 9990 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:35853
I20260811 06:23:19.865445 10133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:35853 every 8 connection(s)
I20260811 06:23:19.866708 9990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:19.872975 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 9990
I20260811 06:23:19.873199 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:19.874054 7186 external_mini_cluster.cc:1428] Setting key 880f4492975cc6b226fe7199ea627cfe
Aug 11 06:23:19 dist-test-slave-07c2 krb5kdc[9877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429399, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:19.889562 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:19.869438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:51065 (local address 127.7.4.190:36249)
0811 06:23:19.869907 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:19.869912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:19.870875 (+ 963us) server_negotiation.cc:407] Connection header received
0811 06:23:19.873003 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:19.873008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:19.873108 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:19.873222 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:19.875203 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.876156 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.877755 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.878653 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.878820 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:19.882783 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:19.882793 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:19.882800 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:19.882846 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:19.885812 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:19.886793 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:19.886801 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:19.886804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:19.887022 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:19.887616 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:19.887623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:19.887628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:19.888038 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:19.888168 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:19.889103 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 06:23:19.889301 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
I20260811 06:23:19.891266 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:19.869993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36249 (local address 127.7.4.129:51065)
0811 06:23:19.870705 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:19.870753 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:19.872635 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:19.873506 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:19.873520 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:19.874325 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:19.875022 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:19.875040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.876329 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.876333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:19.877583 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:19.877596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:19.879012 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:19.879023 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:19.879367 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:19.880096 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:19.880120 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:19.882552 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:19.885961 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:19.885972 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:19.885991 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:19.886649 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:19.887172 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:19.887180 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:19.887186 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:19.887476 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:19.888187 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:19.888197 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:19.888663 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 06:23:19.889010 (+ 347us) client_negotiation.cc:240] Negotiation successful
0811 06:23:19.889741 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":286,"threads_started":1}
I20260811 06:23:19.893010 10134 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36249
I20260811 06:23:19.893385 10134 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:19.894272 10134 heartbeater.cc:511] Master 127.7.4.190:36249 requested a full tablet report, sending...
I20260811 06:23:19.896541 9927 ts_manager.cc:194] Registered new tserver with Master: c2e5917561bf43679783331a0ebfef32 (127.7.4.129:35853)
I20260811 06:23:19.897961 9927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:51065
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:19.929080 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:20.277259 10140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:20.277539 10140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:20.277717 10140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:20.285152 10140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:20.285284 10140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:20.285460 10140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:20.295281 10140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:20.296612 10140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:20.297968 10140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:20.311533 10145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:20.311596 10148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:20.311655 10146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:20.312244 10140 server_base.cc:1034] running on GCE node
I20260811 06:23:20.312928 10140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:20.313889 10140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:20.315147 10140 hybrid_clock.cc:648] HybridClock initialized: now 1786429400315091 us; error 43 us; skew 500 ppm
I20260811 06:23:20.315531 10140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:20 dist-test-slave-07c2 krb5kdc[9877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:20.318720 10140 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:20.320245 10140 webserver.cc:460] Webserver started at http://127.7.4.130:45329/ using document root <none> and password file <none>
I20260811 06:23:20.320962 10140 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:20.321106 10140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:20.321424 10140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:20.324409 10140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "fc5046057ef5480592daabbefa9ebb56"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "07730ab92a08390edac118009c005168"
server_key_iv: "90c2488c1c14c64db0a18a445ca48f11"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.325202 10140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "fc5046057ef5480592daabbefa9ebb56"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "07730ab92a08390edac118009c005168"
server_key_iv: "90c2488c1c14c64db0a18a445ca48f11"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.330212 10140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:20.333624 10155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.334896 10140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:23:20.335060 10140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "fc5046057ef5480592daabbefa9ebb56"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "07730ab92a08390edac118009c005168"
server_key_iv: "90c2488c1c14c64db0a18a445ca48f11"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.335286 10140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:20.352396 10140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:20.356385 10140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:20.356678 10140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:20.357695 10140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:20.359102 10140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:20.359205 10140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.359366 10140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:20.359437 10140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.393606 10140 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:36609
I20260811 06:23:20.393697 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:36609 every 8 connection(s)
I20260811 06:23:20.394907 10140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:20.402087 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10140
I20260811 06:23:20.402266 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:20.402707 7186 external_mini_cluster.cc:1428] Setting key 2d59209300221324f0eb322ab62a7b42
Aug 11 06:23:20 dist-test-slave-07c2 krb5kdc[9877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:20.414903 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:20.397433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:37933 (local address 127.7.4.190:36249)
0811 06:23:20.397926 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:20.397931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:20.398730 (+ 799us) server_negotiation.cc:407] Connection header received
0811 06:23:20.400633 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:20.400638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:20.400727 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:20.400844 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:20.402307 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.403115 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.404918 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.405400 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.405536 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:20.409046 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:20.409051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:20.409054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:20.409092 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:20.411834 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:20.412549 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:20.412554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:20.412556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:20.412685 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:20.413122 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:20.413126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:20.413128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:20.413404 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:20.413564 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:20.414245 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 06:23:20.414472 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":285,"threads_started":1}
I20260811 06:23:20.415948 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:20.397859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36249 (local address 127.7.4.130:37933)
0811 06:23:20.398572 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:20.398618 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:20.400372 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:20.401079 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:20.401087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:20.401589 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:20.402113 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:20.402129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.403278 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.403282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:20.404779 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:20.404791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.405639 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.405646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:20.405870 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:20.406584 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:20.406607 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:20.408821 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:20.411972 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:20.411981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:20.411996 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:20.412418 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:20.412794 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:20.412799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:20.412803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:20.413001 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:20.413536 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:20.413543 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:20.413883 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 06:23:20.414143 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 06:23:20.414716 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":290,"threads_started":1}
I20260811 06:23:20.417444 10278 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36249
I20260811 06:23:20.417778 10278 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:20.418742 10278 heartbeater.cc:511] Master 127.7.4.190:36249 requested a full tablet report, sending...
I20260811 06:23:20.420437 9927 ts_manager.cc:194] Registered new tserver with Master: fc5046057ef5480592daabbefa9ebb56 (127.7.4.130:36609)
I20260811 06:23:20.421736 9927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:37933
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:20.455776 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:20.802007 10287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:20.802328 10287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:20.802491 10287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:20.810150 10287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:20.810277 10287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:20.810454 10287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:20.820983 10287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:20.822356 10287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:20.823829 10287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:20.837540 10292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:20.837450 10295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:20.837481 10293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:20.838819 10287 server_base.cc:1034] running on GCE node
I20260811 06:23:20.839578 10287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:20.840665 10287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:20.841884 10287 hybrid_clock.cc:648] HybridClock initialized: now 1786429400841819 us; error 60 us; skew 500 ppm
I20260811 06:23:20.842250 10287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:20 dist-test-slave-07c2 krb5kdc[9877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:20.846092 10287 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:20.847837 10287 webserver.cc:460] Webserver started at http://127.7.4.131:38359/ using document root <none> and password file <none>
I20260811 06:23:20.848630 10287 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:20.848789 10287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:20.849156 10287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:20.852556 10287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "098077ff3e7e4754aa2c642b3c605610"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "cbb01eb5d860a7649f7ceee4d8781625"
server_key_iv: "ac4d6e47a8f4f1ce0910cdbb4bdf1c9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.853475 10287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "098077ff3e7e4754aa2c642b3c605610"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "cbb01eb5d860a7649f7ceee4d8781625"
server_key_iv: "ac4d6e47a8f4f1ce0910cdbb4bdf1c9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.859359 10287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:23:20.863094 10302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.864586 10287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:20.864837 10287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "098077ff3e7e4754aa2c642b3c605610"
format_stamp: "Formatted at 2026-08-11 06:23:20 on dist-test-slave-07c2"
server_key: "cbb01eb5d860a7649f7ceee4d8781625"
server_key_iv: "ac4d6e47a8f4f1ce0910cdbb4bdf1c9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:20.865085 10287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:20.901237 10134 heartbeater.cc:503] Master 127.7.4.190:36249 was elected leader, sending a full tablet report...
I20260811 06:23:20.901552 10287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:20.906808 10287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:20.907115 10287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:20.908205 10287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:20.909765 10287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:20.909868 10287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.910022 10287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:20.910104 10287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:20.945417 10287 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46577
I20260811 06:23:20.945503 10422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46577 every 8 connection(s)
I20260811 06:23:20.946794 10287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:20.949949 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10287
I20260811 06:23:20.950141 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:20.950798 7186 external_mini_cluster.cc:1428] Setting key e19a349ff24a8d4eb556c4cef2523c0f
Aug 11 06:23:20 dist-test-slave-07c2 krb5kdc[9877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:20.968251 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:20.949396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:59825 (local address 127.7.4.190:36249)
0811 06:23:20.949889 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:20.949893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:20.951246 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 06:23:20.953826 (+ 2580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:20.953831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:20.953915 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:20.954009 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:20.955600 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.956353 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.957952 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.958410 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.958514 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:20.961935 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:20.961940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:20.961943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:20.961975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:20.964836 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:20.965819 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:20.965824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:20.965828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:20.965988 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:20.966500 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:20.966506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:20.966509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:20.966881 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:20.967006 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:20.967770 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 06:23:20.967963 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":262,"threads_started":1}
I20260811 06:23:20.969367 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:20.950119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36249 (local address 127.7.4.131:59825)
0811 06:23:20.951071 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:20.951119 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:20.953510 (+ 2391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:20.954257 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:20.954265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:20.954804 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:20.955451 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:20.955470 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.956495 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.956499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:20.957769 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:20.957782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:20.958638 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:20.958644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:20.958841 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:20.959524 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:20.959573 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:20.961740 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:20.965013 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:20.965020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:20.965036 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:20.965683 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:20.966105 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:20.966108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:20.966111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:20.966374 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:20.967002 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:20.967009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:20.967400 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 06:23:20.967658 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 06:23:20.968064 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":416,"threads_started":1}
I20260811 06:23:20.970821 10423 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36249
I20260811 06:23:20.971092 10423 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:20.971843 10423 heartbeater.cc:511] Master 127.7.4.190:36249 requested a full tablet report, sending...
I20260811 06:23:20.973181 9927 ts_manager.cc:194] Registered new tserver with Master: 098077ff3e7e4754aa2c642b3c605610 (127.7.4.131:46577)
I20260811 06:23:20.973990 9927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:59825
I20260811 06:23:20.978696 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:23:21.017983 10426 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:21.015499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.7.4.190:36249)
0811 06:23:21.015756 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:21.015766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:21.016156 (+ 390us) server_negotiation.cc:407] Connection header received
0811 06:23:21.016320 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:21.016324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:21.016423 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:21.016514 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:21.017690 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52908: BlockingRecv error: recv got EOF from 127.0.0.1:52908 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:23:21.021204 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9990
W20260811 06:23:21.024050 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9990: Not authorized: Client connection negotiation failed: client connection to 127.7.4.129:35853: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:23:21.024317 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9990
I20260811 06:23:21.046916 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10140
W20260811 06:23:21.049742 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10140: Not authorized: Client connection negotiation failed: client connection to 127.7.4.130:36609: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:23:21.049916 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10140
I20260811 06:23:21.070230 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10287
W20260811 06:23:21.073441 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10287: Not authorized: Client connection negotiation failed: client connection to 127.7.4.131:46577: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:23:21.073639 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10287
I20260811 06:23:21.093575 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 9892
I20260811 06:23:21.094892 9958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:21.201200 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 9892
2026-08-11T06:23:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3895 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:21 dist-test-slave-07c2 krb5kdc[10443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:21 dist-test-slave-07c2 krb5kdc[10443](info): set up 2 sockets
Aug 11 06:23:21 dist-test-slave-07c2 krb5kdc[10443](info): commencing operation
krb5kdc: starting...
W20260811 06:23:22.371826 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:22 dist-test-slave-07c2 krb5kdc[10443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:22.595009 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37653
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:22.919082 10456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:22.919407 10456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:22.919577 10456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:22.926476 10456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:22.926587 10456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:22.926671 10456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:22.926788 10456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:22.926852 10456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:22.936295 10456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:22.937454 10456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:22.938696 10456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:22.950135 10462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:22.950162 10461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:22.950076 10464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:22.951103 10456 server_base.cc:1034] running on GCE node
I20260811 06:23:22.951882 10456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:22.953111 10456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:22.954303 10456 hybrid_clock.cc:648] HybridClock initialized: now 1786429402954251 us; error 44 us; skew 500 ppm
I20260811 06:23:22.954699 10456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:22 dist-test-slave-07c2 krb5kdc[10443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429402, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:22.961104 10456 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:22.962525 10456 webserver.cc:460] Webserver started at http://127.7.4.190:39697/ using document root <none> and password file <none>
I20260811 06:23:22.963169 10456 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:22.963342 10456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:22.963677 10456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:22.966552 10456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "e86765aaf5a940f0a62d8ce839159abd"
format_stamp: "Formatted at 2026-08-11 06:23:22 on dist-test-slave-07c2"
server_key: "9cbb170139113eea31924e30360c1cef"
server_key_iv: "fd8a28c0bd234af4d9c1330c01c0160e"
server_key_version: "encryptionkey@0"
I20260811 06:23:22.967365 10456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "e86765aaf5a940f0a62d8ce839159abd"
format_stamp: "Formatted at 2026-08-11 06:23:22 on dist-test-slave-07c2"
server_key: "9cbb170139113eea31924e30360c1cef"
server_key_iv: "fd8a28c0bd234af4d9c1330c01c0160e"
server_key_version: "encryptionkey@0"
I20260811 06:23:22.972215 10456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:23:22.975646 10471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:22.976876 10456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:23:22.977046 10456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "e86765aaf5a940f0a62d8ce839159abd"
format_stamp: "Formatted at 2026-08-11 06:23:22 on dist-test-slave-07c2"
server_key: "9cbb170139113eea31924e30360c1cef"
server_key_iv: "fd8a28c0bd234af4d9c1330c01c0160e"
server_key_version: "encryptionkey@0"
I20260811 06:23:22.977222 10456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:22.987968 10456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:22.992046 10456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:22.992300 10456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:23.010490 10456 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:37653
I20260811 06:23:23.010524 10532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:37653 every 8 connection(s)
I20260811 06:23:23.011906 10456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:23.015522 10533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:23.015861 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10456
I20260811 06:23:23.016079 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:23.016737 7186 external_mini_cluster.cc:1428] Setting key b6913d2b133b14c01bb8641a1c2636c5
I20260811 06:23:23.029587 10533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd: Bootstrap starting.
Aug 11 06:23:23 dist-test-slave-07c2 krb5kdc[10443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:23.032554 10533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:23.033890 10533 log.cc:824] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:23.037200 10533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd: No bootstrap required, opened a new log
I20260811 06:23:23.041312 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:23.018455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.7.4.190:37653)
0811 06:23:23.019333 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:23.019347 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:23.019411 (+ 64us) server_negotiation.cc:407] Connection header received
0811 06:23:23.022029 (+ 2618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:23.022056 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:23.022534 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:23.022975 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:23.024345 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:23.026355 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:23.027646 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:23.028696 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:23.028883 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:23.032714 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:23.032725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:23.032743 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:23.032793 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:23.036823 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:23.037538 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:23.037545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:23.037554 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:23.037785 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:23.038170 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:23.038174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:23.038177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:23.038859 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:23.039170 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:23.039552 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 06:23:23.039997 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":377,"threads_started":1}
I20260811 06:23:23.044958 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:23:23.046241 10533 raft_consensus.cc:348] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } }
I20260811 06:23:23.046571 10533 raft_consensus.cc:374] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:23.046680 10533 raft_consensus.cc:729] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86765aaf5a940f0a62d8ce839159abd, State: Initialized, Role: FOLLOWER
I20260811 06:23:23.047235 10533 consensus_queue.cc:260] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } }
I20260811 06:23:23.047511 10533 raft_consensus.cc:388] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:23.047640 10533 raft_consensus.cc:482] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:23.047803 10533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:23.049909 10533 raft_consensus.cc:504] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } }
I20260811 06:23:23.050369 10533 leader_election.cc:302] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e86765aaf5a940f0a62d8ce839159abd; no voters:
I20260811 06:23:23.051052 10533 leader_election.cc:288] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:23.051155 10540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:23.052498 10540 raft_consensus.cc:686] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [term 1 LEADER]: Becoming Leader. State: Replica: e86765aaf5a940f0a62d8ce839159abd, State: Running, Role: LEADER
I20260811 06:23:23.053089 10540 consensus_queue.cc:237] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } }
I20260811 06:23:23.053385 10533 sys_catalog.cc:564] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:23.056118 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e86765aaf5a940f0a62d8ce839159abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } } }
I20260811 06:23:23.056566 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:23.060575 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e86765aaf5a940f0a62d8ce839159abd. Latest consensus state: current_term: 1 leader_uuid: "e86765aaf5a940f0a62d8ce839159abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86765aaf5a940f0a62d8ce839159abd" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37653 } } }
I20260811 06:23:23.061074 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:23.063093 10547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:23.069329 10547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:23.089390 10547 catalog_manager.cc:1347] Generated new cluster ID: d40b482fe2b345a78f45dd725216447f
I20260811 06:23:23.089556 10547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:23.107201 10547 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:23.107312 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:23.097227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.7.4.190:37653)
0811 06:23:23.097468 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:23.097479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:23.099907 (+ 2428us) server_negotiation.cc:407] Connection header received
0811 06:23:23.100120 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:23.100136 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:23.100199 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:23.100298 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:23.101364 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:23.102119 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:23.103465 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:23.103902 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:23.104039 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:23.105203 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:23.105210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:23.105215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:23.106293 (+ 1078us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:23.106383 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:23.106413 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:23.106538 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:23.106710 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 06:23:23.106806 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":69}
I20260811 06:23:23.108928 10547 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:23.115464 10547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e86765aaf5a940f0a62d8ce839159abd: Generated new TSK 0
I20260811 06:23:23.116452 10547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:23:23.124137 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10456
I20260811 06:23:23.125681 10527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:23.214534 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10456
2026-08-11T06:23:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2006 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:23 dist-test-slave-07c2 krb5kdc[10573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:23 dist-test-slave-07c2 krb5kdc[10573](info): set up 2 sockets
Aug 11 06:23:23 dist-test-slave-07c2 krb5kdc[10573](info): commencing operation
krb5kdc: starting...
W20260811 06:23:24.365815 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:24 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:24.592772 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:35129
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:35129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:24.918838 10588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:24.919189 10588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:24.919386 10588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:24.926537 10588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:24.926676 10588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:24.926823 10588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:24.926939 10588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:24.927022 10588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:24.936857 10588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:35129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:35129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:24.938236 10588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:24.939627 10588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:24.951054 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:24.951102 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:24.951012 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:24.951934 10588 server_base.cc:1034] running on GCE node
I20260811 06:23:24.952713 10588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:24.954072 10588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:24.955332 10588 hybrid_clock.cc:648] HybridClock initialized: now 1786429404955254 us; error 29 us; skew 500 ppm
I20260811 06:23:24.955691 10588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:24 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:24.958683 10588 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:24.960115 10588 webserver.cc:460] Webserver started at http://127.7.4.190:46879/ using document root <none> and password file <none>
I20260811 06:23:24.960752 10588 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:24.960851 10588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:24.961161 10588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:24.964195 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "ce724ae215804c57be605d206a1f38a3"
format_stamp: "Formatted at 2026-08-11 06:23:24 on dist-test-slave-07c2"
server_key: "766416dd44e3b8587ec4a9ceb403830e"
server_key_iv: "92d252e448832866f823de418eb495f9"
server_key_version: "encryptionkey@0"
I20260811 06:23:24.964936 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "ce724ae215804c57be605d206a1f38a3"
format_stamp: "Formatted at 2026-08-11 06:23:24 on dist-test-slave-07c2"
server_key: "766416dd44e3b8587ec4a9ceb403830e"
server_key_iv: "92d252e448832866f823de418eb495f9"
server_key_version: "encryptionkey@0"
I20260811 06:23:24.969913 10588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:23:24.973162 10607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:24.974356 10588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:24.974521 10588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "ce724ae215804c57be605d206a1f38a3"
format_stamp: "Formatted at 2026-08-11 06:23:24 on dist-test-slave-07c2"
server_key: "766416dd44e3b8587ec4a9ceb403830e"
server_key_iv: "92d252e448832866f823de418eb495f9"
server_key_version: "encryptionkey@0"
I20260811 06:23:24.974707 10588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:24.992883 10588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:24.996655 10588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:24.996946 10588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:25.015558 10588 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:35129
I20260811 06:23:25.015599 10668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:35129 every 8 connection(s)
I20260811 06:23:25.016925 10588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:25.020619 10669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:25.023362 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10588
I20260811 06:23:25.023600 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:25.024211 7186 external_mini_cluster.cc:1428] Setting key 5c4e3cf76ec9927254ee83e49e29a924
I20260811 06:23:25.033963 10669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3: Bootstrap starting.
I20260811 06:23:25.037581 10669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:25.038566 10669 log.cc:824] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:23:25 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:25.041512 10669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3: No bootstrap required, opened a new log
I20260811 06:23:25.047940 10672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:25.026246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.7.4.190:35129)
0811 06:23:25.027022 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:25.027037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:25.027101 (+ 64us) server_negotiation.cc:407] Connection header received
0811 06:23:25.029575 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:25.029607 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:25.030144 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:25.030627 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:25.032032 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.034014 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.035246 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.036363 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.036543 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:25.040620 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:25.040628 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:25.040644 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:25.040682 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:25.043866 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:25.044559 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:25.044566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:25.044571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:25.044732 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:25.045107 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:25.045113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:25.045116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:25.045641 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:25.045990 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:25.046429 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 06:23:25.046836 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":304,"threads_started":1}
I20260811 06:23:25.049518 10669 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } }
I20260811 06:23:25.049988 10669 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:25.050110 10669 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce724ae215804c57be605d206a1f38a3, State: Initialized, Role: FOLLOWER
I20260811 06:23:25.050722 10669 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } }
I20260811 06:23:25.051045 10669 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:25.051237 10669 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:25.051429 10669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:25.053725 10669 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } }
I20260811 06:23:25.054262 10669 leader_election.cc:302] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce724ae215804c57be605d206a1f38a3; no voters:
I20260811 06:23:25.054895 10669 leader_election.cc:288] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:25.055032 10674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:25.055799 10674 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [term 1 LEADER]: Becoming Leader. State: Replica: ce724ae215804c57be605d206a1f38a3, State: Running, Role: LEADER
I20260811 06:23:25.056315 10674 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } }
I20260811 06:23:25.057070 10669 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:25.059450 10676 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce724ae215804c57be605d206a1f38a3. Latest consensus state: current_term: 1 leader_uuid: "ce724ae215804c57be605d206a1f38a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } } }
I20260811 06:23:25.059576 10675 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce724ae215804c57be605d206a1f38a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce724ae215804c57be605d206a1f38a3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35129 } } }
I20260811 06:23:25.059898 10676 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:25.059902 10675 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:25.061460 10683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:25.066016 10683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:25.074435 10683 catalog_manager.cc:1347] Generated new cluster ID: a5bf4efd8d2645e8919713e06fc36543
I20260811 06:23:25.074576 10683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:25.097450 10683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:25.098815 10683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:25.110401 10683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce724ae215804c57be605d206a1f38a3: Generated new TSK 0
I20260811 06:23:25.111212 10683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:25.171154 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:25.514719 10696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:25.514982 10696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:25.515148 10696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:25.522235 10696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:25.522337 10696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:25.522526 10696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:25.532502 10696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:25.533915 10696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:25.535218 10696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:25.548609 10702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:25.548851 10704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:25.548626 10701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:25.549638 10696 server_base.cc:1034] running on GCE node
I20260811 06:23:25.550256 10696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:25.551149 10696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:25.552346 10696 hybrid_clock.cc:648] HybridClock initialized: now 1786429405552293 us; error 49 us; skew 500 ppm
I20260811 06:23:25.552654 10696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:25 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429405, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:25.556067 10696 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:25.557505 10696 webserver.cc:460] Webserver started at http://127.7.4.129:33839/ using document root <none> and password file <none>
I20260811 06:23:25.558231 10696 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:25.558334 10696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:25.558601 10696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:25.561784 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "6e590415124b40a1bb81bec5e7bfc61c"
format_stamp: "Formatted at 2026-08-11 06:23:25 on dist-test-slave-07c2"
server_key: "d3fe110555b2a5b7aa9208c05ecb5c23"
server_key_iv: "23a73e3001da1ba90e35135efe5d13d7"
server_key_version: "encryptionkey@0"
I20260811 06:23:25.562705 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "6e590415124b40a1bb81bec5e7bfc61c"
format_stamp: "Formatted at 2026-08-11 06:23:25 on dist-test-slave-07c2"
server_key: "d3fe110555b2a5b7aa9208c05ecb5c23"
server_key_iv: "23a73e3001da1ba90e35135efe5d13d7"
server_key_version: "encryptionkey@0"
I20260811 06:23:25.567961 10696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 06:23:25.571264 10711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:25.572638 10696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:25.572819 10696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "6e590415124b40a1bb81bec5e7bfc61c"
format_stamp: "Formatted at 2026-08-11 06:23:25 on dist-test-slave-07c2"
server_key: "d3fe110555b2a5b7aa9208c05ecb5c23"
server_key_iv: "23a73e3001da1ba90e35135efe5d13d7"
server_key_version: "encryptionkey@0"
I20260811 06:23:25.573135 10696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:25.595778 10696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:25.600493 10696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:25.600811 10696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:25.601771 10696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:25.603124 10696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:25.603235 10696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:25.603403 10696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:25.603474 10696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:25.637271 10696 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:44751
I20260811 06:23:25.637323 10835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:44751 every 8 connection(s)
I20260811 06:23:25.638561 10696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:25.645699 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10696
I20260811 06:23:25.645960 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:25.646808 7186 external_mini_cluster.cc:1428] Setting key f9d43b2f7f988f9d80b822ea74e17609
Aug 11 06:23:25 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429405, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:25.661806 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:25.641222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:35617 (local address 127.7.4.190:35129)
0811 06:23:25.641703 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:25.641708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:25.642659 (+ 951us) server_negotiation.cc:407] Connection header received
0811 06:23:25.644557 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:25.644562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:25.644656 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:25.644771 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:25.646825 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.647795 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.649880 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.650269 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.650375 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:25.655142 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:25.655151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:25.655159 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:25.655208 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:25.658652 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:25.659439 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:25.659449 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:25.659452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:25.659568 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:25.660014 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:25.660019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:25.660027 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:25.660293 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:25.660458 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:25.661190 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 06:23:25.661412 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":219,"threads_started":1}
I20260811 06:23:25.662977 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:25.641774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35129 (local address 127.7.4.129:35617)
0811 06:23:25.642494 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:25.642532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:25.644280 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:25.645007 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:25.645014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:25.645791 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:25.646638 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:25.646665 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.647959 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.647963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:25.649729 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:25.649747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:25.650525 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:25.650534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:25.650902 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:25.651759 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:25.651800 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:25.654870 (+ 3070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:25.658802 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:25.658812 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:25.658830 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:25.659274 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:25.659686 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:25.659690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:25.659694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:25.659894 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:25.660432 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:25.660440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:25.660795 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 06:23:25.661060 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 06:23:25.661612 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":287,"threads_started":1}
I20260811 06:23:25.664537 10836 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35129
I20260811 06:23:25.664845 10836 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:25.665584 10836 heartbeater.cc:511] Master 127.7.4.190:35129 requested a full tablet report, sending...
I20260811 06:23:25.667768 10626 ts_manager.cc:194] Registered new tserver with Master: 6e590415124b40a1bb81bec5e7bfc61c (127.7.4.129:44751)
I20260811 06:23:25.669229 10626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:35617
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:25.701979 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:26.037057 10843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:26.037319 10843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:26.037488 10843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:26.044891 10843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:26.045023 10843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:26.045212 10843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:26.054823 10843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:26.056178 10843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:26.057520 10843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:26.070721 10848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:26.070866 10849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:26.070814 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:26.071856 10843 server_base.cc:1034] running on GCE node
I20260811 06:23:26.072465 10843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:26.073452 10843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:26.074682 10843 hybrid_clock.cc:648] HybridClock initialized: now 1786429406074623 us; error 65 us; skew 500 ppm
I20260811 06:23:26.074999 10843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:26.078202 10843 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:26.079754 10843 webserver.cc:460] Webserver started at http://127.7.4.130:43081/ using document root <none> and password file <none>
I20260811 06:23:26.080447 10843 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:26.080583 10843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:26.080900 10843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:26.083895 10843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "8d09b714f9b745baaf89a8d9a8089469"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "70d4a4b3c1f5ae50deb7fe3cec2156bb"
server_key_iv: "8c268d7697915006d0eb8b21ed628d9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.084694 10843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "8d09b714f9b745baaf89a8d9a8089469"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "70d4a4b3c1f5ae50deb7fe3cec2156bb"
server_key_iv: "8c268d7697915006d0eb8b21ed628d9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.089766 10843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:23:26.093184 10859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.094389 10843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:23:26.094558 10843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "8d09b714f9b745baaf89a8d9a8089469"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "70d4a4b3c1f5ae50deb7fe3cec2156bb"
server_key_iv: "8c268d7697915006d0eb8b21ed628d9b"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.094774 10843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:26.133862 10843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:26.137887 10843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:26.138209 10843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:26.139181 10843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:26.140573 10843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:26.140681 10843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.140828 10843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:26.140923 10843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.175086 10843 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:44115
I20260811 06:23:26.175129 10984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:44115 every 8 connection(s)
I20260811 06:23:26.176507 10843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:26.184895 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10843
I20260811 06:23:26.185091 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:26.185635 7186 external_mini_cluster.cc:1428] Setting key 5afe8e99ebdf847af49dd416c60b7c91
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:26.197312 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.179369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:35869 (local address 127.7.4.190:35129)
0811 06:23:26.179846 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.179850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.180707 (+ 857us) server_negotiation.cc:407] Connection header received
0811 06:23:26.182482 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.182487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.182577 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.182693 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.184266 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.185303 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.187283 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.187750 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.187886 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.191700 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.191705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.191708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.191739 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.194231 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.194943 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.194966 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.194969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.195095 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.195592 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.195596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.195598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.195865 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.196033 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.196718 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.196947 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260811 06:23:26.198395 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.179869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35129 (local address 127.7.4.130:35869)
0811 06:23:26.180543 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:26.180587 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:26.182202 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:26.182926 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:26.182934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:26.183512 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:26.184104 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:26.184119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.185481 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.185485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:26.187119 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:26.187133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.187978 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.187985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:26.188216 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.188957 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:26.188979 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:26.191471 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:26.194376 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:26.194385 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:26.194400 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:26.194814 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:26.195209 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:26.195214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:26.195217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:26.195469 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:26.196003 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:26.196011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:26.196345 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 06:23:26.196599 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 06:23:26.197124 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":285,"threads_started":1}
I20260811 06:23:26.199885 10985 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35129
I20260811 06:23:26.200210 10985 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:26.201092 10985 heartbeater.cc:511] Master 127.7.4.190:35129 requested a full tablet report, sending...
I20260811 06:23:26.202780 10626 ts_manager.cc:194] Registered new tserver with Master: 8d09b714f9b745baaf89a8d9a8089469 (127.7.4.130:44115)
I20260811 06:23:26.204114 10626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:35869
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:26.238433 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:26.610025 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:26.610296 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:26.610482 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:26.617646 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:26.617766 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:26.617935 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:26.627358 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:26.628640 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:26.629928 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:26.644138 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:26.644214 10999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:26.644153 10996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:26.644912 10991 server_base.cc:1034] running on GCE node
I20260811 06:23:26.645486 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:26.646375 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:26.647573 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786429406647507 us; error 50 us; skew 500 ppm
I20260811 06:23:26.647883 10991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:26.651139 10991 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:26.652649 10991 webserver.cc:460] Webserver started at http://127.7.4.131:36433/ using document root <none> and password file <none>
I20260811 06:23:26.653295 10991 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:26.653394 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:26.653644 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:26.656522 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "0849dee5a9b54c02a4b4f498351dfc07"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "b953f823b89a4f64c8551b528afd1cb4"
server_key_iv: "74868257c3bd16b46517f2b9067aecb3"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.657326 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "0849dee5a9b54c02a4b4f498351dfc07"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "b953f823b89a4f64c8551b528afd1cb4"
server_key_iv: "74868257c3bd16b46517f2b9067aecb3"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.662919 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:26.666338 11007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.667765 10991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:26.667943 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "0849dee5a9b54c02a4b4f498351dfc07"
format_stamp: "Formatted at 2026-08-11 06:23:26 on dist-test-slave-07c2"
server_key: "b953f823b89a4f64c8551b528afd1cb4"
server_key_iv: "74868257c3bd16b46517f2b9067aecb3"
server_key_version: "encryptionkey@0"
I20260811 06:23:26.668140 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:26.672163 10836 heartbeater.cc:503] Master 127.7.4.190:35129 was elected leader, sending a full tablet report...
I20260811 06:23:26.690304 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:26.694624 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:26.695081 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:26.696197 10991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:26.697505 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:26.697604 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.697755 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:26.697806 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:26.731977 10991 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:43793
I20260811 06:23:26.732026 11131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:43793 every 8 connection(s)
I20260811 06:23:26.733299 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:26.741607 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 10991
I20260811 06:23:26.741814 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:26.742336 7186 external_mini_cluster.cc:1428] Setting key 9379d20992b0654ee27f3178a0d7369e
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:26.754117 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.735989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:50779 (local address 127.7.4.190:35129)
0811 06:23:26.736466 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.736470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.737367 (+ 897us) server_negotiation.cc:407] Connection header received
0811 06:23:26.739163 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.739167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.739235 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.739394 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.740840 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.741963 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.743801 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.744413 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.744536 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.748445 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.748452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.748458 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.748501 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.751436 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.752178 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.752182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.752184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.752311 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.752690 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.752693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.752695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.752975 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.753095 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.753698 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.753852 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":224,"threads_started":1}
I20260811 06:23:26.755085 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.736512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35129 (local address 127.7.4.131:50779)
0811 06:23:26.737204 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:26.737249 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:26.738900 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:26.739625 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:26.739634 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:26.740121 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:26.740667 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:26.740685 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.742136 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.742141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:26.743653 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:26.743671 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.744685 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.744694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:26.744991 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.745728 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:26.745760 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:26.748158 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:26.751551 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:26.751557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:26.751568 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:26.752055 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:26.752408 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:26.752411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:26.752413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:26.752593 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:26.753077 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:26.753082 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:26.753370 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 06:23:26.753654 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:23:26.754029 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260811 06:23:26.756829 11132 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35129
I20260811 06:23:26.757153 11132 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:26.758039 11132 heartbeater.cc:511] Master 127.7.4.190:35129 requested a full tablet report, sending...
I20260811 06:23:26.759495 10626 ts_manager.cc:194] Registered new tserver with Master: 0849dee5a9b54c02a4b4f498351dfc07 (127.7.4.131:43793)
I20260811 06:23:26.760277 10626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:50779
I20260811 06:23:26.769775 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:26.824584 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.810162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.7.4.190:35129)
0811 06:23:26.810391 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.810401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.810774 (+ 373us) server_negotiation.cc:407] Connection header received
0811 06:23:26.810989 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.810993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.811063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.811169 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.812591 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.813527 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.814812 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.815248 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.815422 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.819501 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.819505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.819509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.819563 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.822260 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.822888 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.822891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.822893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.823022 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.823413 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.823416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.823418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.823735 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.823934 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.824191 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.824352 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 06:23:26.825142 10626 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54308
I20260811 06:23:26.826936 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10696
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:26.842341 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.827830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.7.4.129:44751)
0811 06:23:26.828459 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.828465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.828489 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:23:26.828561 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.828572 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.828768 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.828891 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.830011 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.831024 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.832265 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.832811 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.832907 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.836710 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.836722 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.836734 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.836765 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.839657 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.840307 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.840314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.840317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.840458 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.840829 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.840835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.840838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.841172 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.841481 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.841718 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.841983 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":316,"threads_started":1}
W20260811 06:23:26.843575 10831 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52136
W20260811 06:23:26.844630 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10696: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:26.844791 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10696
I20260811 06:23:26.865000 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10843
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:26.880841 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.865828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.7.4.130:44115)
0811 06:23:26.866464 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.866471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.866496 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:23:26.866555 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.866561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.866783 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.866946 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.868273 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.869327 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.870524 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.871095 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.871237 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.875547 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.875559 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.875572 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.875613 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.878250 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.878877 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.878885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.878887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.879077 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.879453 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.879456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.879458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.879792 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.880015 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.880285 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.880515 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":355,"threads_started":1}
W20260811 06:23:26.882109 10979 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42728
W20260811 06:23:26.883152 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10843: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:26.883352 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10843
I20260811 06:23:26.903424 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10991
Aug 11 06:23:26 dist-test-slave-07c2 krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429406, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:26.919716 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:26.904353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.7.4.131:43793)
0811 06:23:26.904992 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:26.904999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:26.905028 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:23:26.905089 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:26.905095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:26.905293 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:26.905463 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:26.906777 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.907869 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.909196 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:26.909735 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:26.909887 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:26.914072 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:26.914084 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:26.914096 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:26.914132 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:26.917102 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.917766 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.917773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.917775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.917898 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:26.918327 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:26.918331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:26.918333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:26.918696 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:26.918937 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:26.919145 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 06:23:26.919419 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":341,"threads_started":1}
W20260811 06:23:26.920953 11127 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47734
W20260811 06:23:26.922032 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10991: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:23:26.922169 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10991
I20260811 06:23:26.941990 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 10588
I20260811 06:23:26.943056 10662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:27.048038 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 10588
2026-08-11T06:23:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3839 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:27 dist-test-slave-07c2 krb5kdc[11152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:27 dist-test-slave-07c2 krb5kdc[11152](info): set up 2 sockets
Aug 11 06:23:27 dist-test-slave-07c2 krb5kdc[11152](info): commencing operation
krb5kdc: starting...
W20260811 06:23:28.196410 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:28 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:28.412074 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36635
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:28.732057 11166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:28.732374 11166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:28.732522 11166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:28.739488 11166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:28.739615 11166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:28.739722 11166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:28.739816 11166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:28.739888 11166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:28.749253 11166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:28.750470 11166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:28.751833 11166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:28.763204 11172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:28.763317 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:28.763419 11177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:28.764353 11166 server_base.cc:1034] running on GCE node
I20260811 06:23:28.765060 11166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:28.766285 11166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:28.767469 11166 hybrid_clock.cc:648] HybridClock initialized: now 1786429408767424 us; error 29 us; skew 500 ppm
I20260811 06:23:28.767758 11166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:28 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:28.773986 11166 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:28.775408 11166 webserver.cc:460] Webserver started at http://127.7.4.190:41185/ using document root <none> and password file <none>
I20260811 06:23:28.776011 11166 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:28.776111 11166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:28.776435 11166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:28.779711 11166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "2fdd5e148e364d659de87898825e0eb3"
format_stamp: "Formatted at 2026-08-11 06:23:28 on dist-test-slave-07c2"
server_key: "65105403bcc7b12c0889981d5409e7d2"
server_key_iv: "e64b982ec612f5150d8639823f56f658"
server_key_version: "encryptionkey@0"
I20260811 06:23:28.780400 11166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "2fdd5e148e364d659de87898825e0eb3"
format_stamp: "Formatted at 2026-08-11 06:23:28 on dist-test-slave-07c2"
server_key: "65105403bcc7b12c0889981d5409e7d2"
server_key_iv: "e64b982ec612f5150d8639823f56f658"
server_key_version: "encryptionkey@0"
I20260811 06:23:28.785131 11166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:23:28.788269 11184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:28.789486 11166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:28.789669 11166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "2fdd5e148e364d659de87898825e0eb3"
format_stamp: "Formatted at 2026-08-11 06:23:28 on dist-test-slave-07c2"
server_key: "65105403bcc7b12c0889981d5409e7d2"
server_key_iv: "e64b982ec612f5150d8639823f56f658"
server_key_version: "encryptionkey@0"
I20260811 06:23:28.789846 11166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:28.805192 11166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:28.809296 11166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:28.809520 11166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:28.827548 11166 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:36635
I20260811 06:23:28.827581 11237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:36635 every 8 connection(s)
I20260811 06:23:28.828863 11166 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:28.830639 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11166
I20260811 06:23:28.830847 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:28.831714 7186 external_mini_cluster.cc:1428] Setting key 4f3a7e2996ed9b0622a3b2377e23cdf8
I20260811 06:23:28.833248 11238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:23:28 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:28.846957 11238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3: Bootstrap starting.
I20260811 06:23:28.849967 11238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:28.850953 11238 log.cc:824] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:28.854242 11238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3: No bootstrap required, opened a new log
I20260811 06:23:28.854840 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:28.833751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.7.4.190:36635)
0811 06:23:28.834522 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:28.834533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:28.834570 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:23:28.836218 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:28.836238 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:28.836583 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:28.836902 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:28.838824 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:28.840262 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:28.842044 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:28.842643 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:28.842781 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:28.847400 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:28.847409 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:28.847421 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:28.847454 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:28.850486 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:28.851195 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:28.851203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:28.851210 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:28.851442 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:28.851880 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:28.851884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:28.851886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:28.852459 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:28.852739 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:28.853139 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 06:23:28.853536 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":370,"threads_started":1}
I20260811 06:23:28.861778 11238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } }
I20260811 06:23:28.862092 11238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:28.862161 11238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdd5e148e364d659de87898825e0eb3, State: Initialized, Role: FOLLOWER
I20260811 06:23:28.862596 11238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } }
I20260811 06:23:28.862810 11238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:28.862903 11238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:28.863023 11238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:28.865017 11238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } }
I20260811 06:23:28.865422 11238 leader_election.cc:302] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fdd5e148e364d659de87898825e0eb3; no voters:
I20260811 06:23:28.865984 11238 leader_election.cc:288] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:28.866091 11243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:28.867228 11243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 2fdd5e148e364d659de87898825e0eb3, State: Running, Role: LEADER
I20260811 06:23:28.867750 11243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } }
I20260811 06:23:28.868001 11238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:28.870536 11247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fdd5e148e364d659de87898825e0eb3. Latest consensus state: current_term: 1 leader_uuid: "2fdd5e148e364d659de87898825e0eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } } }
I20260811 06:23:28.870988 11247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:28.870549 11246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fdd5e148e364d659de87898825e0eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdd5e148e364d659de87898825e0eb3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36635 } } }
I20260811 06:23:28.871404 11246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:28.876751 11252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:28.881330 11252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:28.888881 11252 catalog_manager.cc:1347] Generated new cluster ID: 19e23fc4f93643bbb4247c525b831252
I20260811 06:23:28.888978 11252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:28.906810 11252 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:28.908239 11252 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:28.918267 11252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fdd5e148e364d659de87898825e0eb3: Generated new TSK 0
I20260811 06:23:28.919042 11252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:28.969262 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:29.294723 11266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:29.294974 11266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:29.295150 11266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:29.303903 11266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:29.304052 11266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:29.304268 11266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:29.314172 11266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:29.315546 11266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:29.316890 11266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:29.329658 11271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:29.329603 11272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:29.329669 11274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:29.330312 11266 server_base.cc:1034] running on GCE node
I20260811 06:23:29.330921 11266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:29.331869 11266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:29.333063 11266 hybrid_clock.cc:648] HybridClock initialized: now 1786429409333007 us; error 55 us; skew 500 ppm
I20260811 06:23:29.333370 11266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:29 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:29.336615 11266 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:29.337946 11266 webserver.cc:460] Webserver started at http://127.7.4.129:35925/ using document root <none> and password file <none>
I20260811 06:23:29.338565 11266 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:29.338667 11266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:29.338917 11266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:29.341732 11266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "c3f0475754a44b6d99a4a7de327ae438"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "251ab23234c16be420d1374a1e6bcb20"
server_key_iv: "e19c9d2c40d809fb81a5d0be08111399"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.342494 11266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "c3f0475754a44b6d99a4a7de327ae438"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "251ab23234c16be420d1374a1e6bcb20"
server_key_iv: "e19c9d2c40d809fb81a5d0be08111399"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.347397 11266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:23:29.350586 11283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.351866 11266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:29.352039 11266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "c3f0475754a44b6d99a4a7de327ae438"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "251ab23234c16be420d1374a1e6bcb20"
server_key_iv: "e19c9d2c40d809fb81a5d0be08111399"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.352240 11266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:29.382936 11266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:29.387391 11266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:29.387683 11266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:29.388667 11266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:29.389959 11266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:29.390084 11266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.390244 11266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:29.390327 11266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.423547 11266 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:33485
I20260811 06:23:29.423588 11407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:33485 every 8 connection(s)
I20260811 06:23:29.424785 11266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:29.429312 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11266
I20260811 06:23:29.429512 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:29.430179 7186 external_mini_cluster.cc:1428] Setting key 0f3098181eeb41ce0afb1d603441e10a
Aug 11 06:23:29 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:29.444410 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:29.427256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:54699 (local address 127.7.4.190:36635)
0811 06:23:29.427700 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:29.427705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:29.428685 (+ 980us) server_negotiation.cc:407] Connection header received
0811 06:23:29.430642 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:29.430647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:29.430739 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:29.430862 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:29.432374 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.433077 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.434590 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.434995 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.435190 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:29.438802 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:29.438807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:29.438810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:29.438840 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:29.441373 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:29.442114 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:29.442119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:29.442121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:29.442244 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:29.442693 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:29.442696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:29.442698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:29.442953 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:29.443120 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:29.443841 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 06:23:29.444066 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":209,"threads_started":1}
I20260811 06:23:29.445314 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:29.427817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36635 (local address 127.7.4.129:54699)
0811 06:23:29.428512 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:29.428570 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:29.430384 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:29.431103 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:29.431110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:29.431686 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:29.432205 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:29.432223 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.433253 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.433257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:29.434431 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:29.434443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.435260 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.435270 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:29.435557 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:29.436234 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:29.436255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:29.438579 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:29.441515 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:29.441523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:29.441536 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:29.441956 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:29.442356 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:29.442362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:29.442366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:29.442561 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:29.443095 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:29.443102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:29.443476 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 06:23:29.443722 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 06:23:29.444185 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":265,"threads_started":1}
I20260811 06:23:29.446835 11408 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36635
I20260811 06:23:29.447140 11408 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:29.448014 11408 heartbeater.cc:511] Master 127.7.4.190:36635 requested a full tablet report, sending...
I20260811 06:23:29.450476 11201 ts_manager.cc:194] Registered new tserver with Master: c3f0475754a44b6d99a4a7de327ae438 (127.7.4.129:33485)
I20260811 06:23:29.452710 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:54699
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:29.481861 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:29.808732 11414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:29.808974 11414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:29.809139 11414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:29.816185 11414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:29.816304 11414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:29.816493 11414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:29.825382 11414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:29.826555 11414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:29.828250 11414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:29.842482 11422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:29.842536 11420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:29.844357 11419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:29.847016 11414 server_base.cc:1034] running on GCE node
I20260811 06:23:29.847618 11414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:29.848439 11414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:29.849665 11414 hybrid_clock.cc:648] HybridClock initialized: now 1786429409849643 us; error 63 us; skew 500 ppm
I20260811 06:23:29.849996 11414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:29 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:29.853039 11414 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:29.854558 11414 webserver.cc:460] Webserver started at http://127.7.4.130:45099/ using document root <none> and password file <none>
I20260811 06:23:29.855230 11414 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:29.855406 11414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:29.855732 11414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:29.858629 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "4f70d1746b38465bb651ed17d764abba"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "09933e8dba0425c5cdc70b0bcc6e341d"
server_key_iv: "9bb78611d40545619e9593290f186997"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.859448 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "4f70d1746b38465bb651ed17d764abba"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "09933e8dba0425c5cdc70b0bcc6e341d"
server_key_iv: "9bb78611d40545619e9593290f186997"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.864385 11414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:23:29.867735 11429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.868988 11414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:23:29.869154 11414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "4f70d1746b38465bb651ed17d764abba"
format_stamp: "Formatted at 2026-08-11 06:23:29 on dist-test-slave-07c2"
server_key: "09933e8dba0425c5cdc70b0bcc6e341d"
server_key_iv: "9bb78611d40545619e9593290f186997"
server_key_version: "encryptionkey@0"
I20260811 06:23:29.869366 11414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:29.883126 11414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:29.886639 11414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:29.886859 11414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:29.887820 11414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:29.889015 11414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:29.889086 11414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.889168 11414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:29.889215 11414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:29.923251 11414 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34519
I20260811 06:23:29.923354 11556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34519 every 8 connection(s)
I20260811 06:23:29.924471 11414 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:29.932325 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11414
I20260811 06:23:29.932536 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:29.933066 7186 external_mini_cluster.cc:1428] Setting key 23b914a7902e0fefe7ed2121e6441e37
Aug 11 06:23:29 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:29.945737 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:29.926964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:48109 (local address 127.7.4.190:36635)
0811 06:23:29.927126 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:29.927131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:29.928330 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 06:23:29.930169 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:29.930175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:29.930274 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:29.930384 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:29.931866 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.933012 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.934799 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.935403 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.935529 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:29.939054 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:29.939063 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:29.939070 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:29.939116 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:29.942663 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:29.943482 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:29.943489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:29.943496 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:29.943658 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:29.944047 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:29.944055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:29.944060 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:29.944422 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:29.944539 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:29.945320 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 06:23:29.945501 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:23:29.947134 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:29.927435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36635 (local address 127.7.4.130:48109)
0811 06:23:29.928173 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:29.928210 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:29.929890 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:29.930611 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:29.930618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:29.931109 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:29.931698 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:29.931716 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.933208 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.933212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:29.934638 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:29.934652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:29.935606 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:29.935612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:29.935802 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:29.936466 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:29.936487 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:29.938829 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:29.942898 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:29.942904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:29.942913 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:29.943371 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:29.943763 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:29.943767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:29.943769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:29.943938 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:29.944549 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:29.944556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:29.944961 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 06:23:29.945233 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 06:23:29.945783 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":297,"threads_started":1}
I20260811 06:23:29.948562 11557 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36635
I20260811 06:23:29.948850 11557 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:29.949534 11557 heartbeater.cc:511] Master 127.7.4.190:36635 requested a full tablet report, sending...
I20260811 06:23:29.950917 11201 ts_manager.cc:194] Registered new tserver with Master: 4f70d1746b38465bb651ed17d764abba (127.7.4.130:34519)
I20260811 06:23:29.951748 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:48109
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:29.984401 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:30.322633 11562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:30.322886 11562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:30.322980 11562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:30.329988 11562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:30.330098 11562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:30.330209 11562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:30.339534 11562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:36635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:30.340662 11562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:30.341886 11562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:30.354722 11571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:30.354658 11568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:30.354749 11567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:30.355665 11562 server_base.cc:1034] running on GCE node
I20260811 06:23:30.356323 11562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:30.357218 11562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:30.358417 11562 hybrid_clock.cc:648] HybridClock initialized: now 1786429410358365 us; error 44 us; skew 500 ppm
I20260811 06:23:30.358747 11562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:30.361904 11562 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:30.363369 11562 webserver.cc:460] Webserver started at http://127.7.4.131:45093/ using document root <none> and password file <none>
I20260811 06:23:30.364017 11562 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:30.364149 11562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:30.364459 11562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:30.367363 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "d7c75c82e45347a3b92b08377970c326"
format_stamp: "Formatted at 2026-08-11 06:23:30 on dist-test-slave-07c2"
server_key: "3675ce0572fd727646dfaa0b816bcccf"
server_key_iv: "7331c6d23ea91a85a40259ff5ef6f0ef"
server_key_version: "encryptionkey@0"
I20260811 06:23:30.368115 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "d7c75c82e45347a3b92b08377970c326"
format_stamp: "Formatted at 2026-08-11 06:23:30 on dist-test-slave-07c2"
server_key: "3675ce0572fd727646dfaa0b816bcccf"
server_key_iv: "7331c6d23ea91a85a40259ff5ef6f0ef"
server_key_version: "encryptionkey@0"
I20260811 06:23:30.373498 11562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:30.377108 11578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:30.378364 11562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:30.378535 11562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "d7c75c82e45347a3b92b08377970c326"
format_stamp: "Formatted at 2026-08-11 06:23:30 on dist-test-slave-07c2"
server_key: "3675ce0572fd727646dfaa0b816bcccf"
server_key_iv: "7331c6d23ea91a85a40259ff5ef6f0ef"
server_key_version: "encryptionkey@0"
I20260811 06:23:30.378757 11562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:30.394608 11562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:30.398731 11562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:30.399019 11562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:30.399996 11562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:30.401217 11562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:30.401314 11562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:30.401463 11562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:30.401533 11562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:30.434929 11562 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:44997
I20260811 06:23:30.434983 11700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:44997 every 8 connection(s)
I20260811 06:23:30.436162 11562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:30.444780 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11562
I20260811 06:23:30.444965 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:30.445519 7186 external_mini_cluster.cc:1428] Setting key 1c5fe42f58d7585c6cf58021ab41e6e5
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:30.456027 11408 heartbeater.cc:503] Master 127.7.4.190:36635 was elected leader, sending a full tablet report...
I20260811 06:23:30.456277 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.438770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:44633 (local address 127.7.4.190:36635)
0811 06:23:30.439007 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.439014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.440203 (+ 1189us) server_negotiation.cc:407] Connection header received
0811 06:23:30.441927 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.441932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.442002 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.442092 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.443558 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.444341 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.446631 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.447277 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.447432 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.450819 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.450824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.450827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.450859 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.453434 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.454085 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.454089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.454091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.454226 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.454593 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.454596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.454598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.454853 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.454962 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.455650 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.455890 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:23:30.457190 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.439220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36635 (local address 127.7.4.131:44633)
0811 06:23:30.440030 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:30.440071 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:30.441676 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:30.442285 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:30.442293 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:30.442811 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:30.443413 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:30.443431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.444509 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.444516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:30.446438 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:30.446451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.447531 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.447539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:30.447774 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.448444 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:30.448465 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:30.450631 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:30.453558 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:30.453564 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:30.453576 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:30.453982 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:30.454320 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:30.454323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:30.454325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:30.454502 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:30.454957 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:30.454962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:30.455273 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 06:23:30.455558 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 06:23:30.455924 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":298,"threads_started":1}
I20260811 06:23:30.458535 11701 heartbeater.cc:347] Connected to a master server at 127.7.4.190:36635
I20260811 06:23:30.458825 11701 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:30.459584 11701 heartbeater.cc:511] Master 127.7.4.190:36635 requested a full tablet report, sending...
I20260811 06:23:30.460963 11201 ts_manager.cc:194] Registered new tserver with Master: d7c75c82e45347a3b92b08377970c326 (127.7.4.131:44997)
I20260811 06:23:30.461725 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:44633
I20260811 06:23:30.472880 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:30.528085 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.513039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.7.4.129:33485)
0811 06:23:30.513759 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.513771 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.513807 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:23:30.513880 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.513887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.514090 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.514275 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.515775 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.516792 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.517999 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.518539 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.518678 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.522657 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.522669 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.522681 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.522727 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.525593 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.526217 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.526229 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.526233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.526402 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.526740 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.526743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.526745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.527055 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.527279 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.527608 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.527802 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":327,"threads_started":1}
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:30.544879 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.532292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.7.4.190:36635)
0811 06:23:30.532506 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.532511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.532551 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:23:30.532731 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.532735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.532800 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.532881 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.533952 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.534627 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.535885 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.536288 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.536380 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.539921 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.539925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.539928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.539959 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.542521 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.543095 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.543099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.543101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.543230 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.543596 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.543599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.543601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.543931 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.544120 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.544375 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.544532 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 06:23:30.545460 11201 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44676
I20260811 06:23:30.546495 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11266
I20260811 06:23:30.559531 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.547486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.7.4.129:33485)
0811 06:23:30.547687 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.547693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.547802 (+ 109us) server_negotiation.cc:407] Connection header received
0811 06:23:30.547932 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.547935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.548018 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.548093 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.549637 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.550299 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.551553 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.551941 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.552031 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.554102 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.554107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.554110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.554140 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.556971 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.557602 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.557609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.557613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.557749 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.558123 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.558128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.558132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.558383 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.558633 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.558956 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.559175 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:30.560127 11403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:30.642530 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11266
I20260811 06:23:30.663362 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11414
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:30.679427 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.664268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.7.4.130:34519)
0811 06:23:30.664837 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.664844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.664869 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:23:30.664929 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.664936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.665144 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.665274 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.666440 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.667582 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.668873 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.669501 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.669594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.673816 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.673827 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.673839 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.673870 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.676905 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.677501 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.677508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.677510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.677629 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.677945 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.677948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.677950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.678264 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.678491 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.678753 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.678979 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":286,"threads_started":1}
I20260811 06:23:30.680647 11552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:30.763362 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11414
I20260811 06:23:30.784834 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11562
Aug 11 06:23:30 dist-test-slave-07c2 krb5kdc[11152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:30.801152 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:30.785893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.7.4.131:44997)
0811 06:23:30.786550 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:30.786557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:30.786586 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:23:30.786666 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:30.786673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:30.786871 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:30.786994 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:30.788325 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.789372 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.790560 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:30.791094 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:30.791186 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:30.795464 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:30.795476 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:30.795488 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:30.795519 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:30.798291 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.798945 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.798951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.798954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.799103 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:30.799598 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:30.799605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:30.799608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:30.799943 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:30.800257 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:30.800571 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:23:30.800825 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":354,"threads_started":1}
I20260811 06:23:30.802386 11696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:30.884347 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11562
I20260811 06:23:30.905324 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11166
I20260811 06:23:30.906487 11233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:31.009852 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11166
2026-08-11T06:23:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3961 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:31 dist-test-slave-07c2 krb5kdc[11723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:31 dist-test-slave-07c2 krb5kdc[11723](info): set up 2 sockets
Aug 11 06:23:31 dist-test-slave-07c2 krb5kdc[11723](info): commencing operation
krb5kdc: starting...
W20260811 06:23:32.160827 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:32 dist-test-slave-07c2 krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:32.381858 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:38791
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:38791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:32.703958 11737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:32.704227 11737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:32.704465 11737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:32.711531 11737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:32.711668 11737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:32.711755 11737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:32.711856 11737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:32.711946 11737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:32.721490 11737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:38791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:38791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:32.722762 11737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:32.724100 11737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:32.735385 11747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:32.735466 11744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:32.735432 11745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:32.736541 11737 server_base.cc:1034] running on GCE node
I20260811 06:23:32.737254 11737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:32.738520 11737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:32.739753 11737 hybrid_clock.cc:648] HybridClock initialized: now 1786429412739722 us; error 51 us; skew 500 ppm
I20260811 06:23:32.740082 11737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:32 dist-test-slave-07c2 krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:32.743046 11737 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:32.744246 11737 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:32.744387 11737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:32.744704 11737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:32.747637 11737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "d00b0cea04984eb2816531c7b82c55e5"
format_stamp: "Formatted at 2026-08-11 06:23:32 on dist-test-slave-07c2"
server_key: "37c8d69fb79524186d2445527d5fd312"
server_key_iv: "7e9c48d95078175831f58e554ba03e6f"
server_key_version: "encryptionkey@0"
I20260811 06:23:32.748435 11737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "d00b0cea04984eb2816531c7b82c55e5"
format_stamp: "Formatted at 2026-08-11 06:23:32 on dist-test-slave-07c2"
server_key: "37c8d69fb79524186d2445527d5fd312"
server_key_iv: "7e9c48d95078175831f58e554ba03e6f"
server_key_version: "encryptionkey@0"
I20260811 06:23:32.753177 11737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 06:23:32.756464 11753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:32.757603 11737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:32.757791 11737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "d00b0cea04984eb2816531c7b82c55e5"
format_stamp: "Formatted at 2026-08-11 06:23:32 on dist-test-slave-07c2"
server_key: "37c8d69fb79524186d2445527d5fd312"
server_key_iv: "7e9c48d95078175831f58e554ba03e6f"
server_key_version: "encryptionkey@0"
I20260811 06:23:32.758018 11737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:32.776453 11737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:32.780382 11737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:32.780653 11737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:32.798925 11737 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:38791
I20260811 06:23:32.798947 11807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:38791 every 8 connection(s)
I20260811 06:23:32.799894 11737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:32.801906 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11737
I20260811 06:23:32.802143 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:32.803035 7186 external_mini_cluster.cc:1428] Setting key 1de2fcb59dbf0e32470e6f785775f938
I20260811 06:23:32.804103 11808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:23:32 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:32.818773 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: Bootstrap starting.
I20260811 06:23:32.821831 11808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:32.822819 11808 log.cc:824] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:32.825573 11808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: No bootstrap required, opened a new log
I20260811 06:23:32.826160 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:32.805238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.7.4.190:38791)
0811 06:23:32.806037 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:32.806048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:32.806086 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:23:32.807689 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:32.807709 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:32.808060 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:32.808365 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:32.810236 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:32.811722 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:32.813519 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:32.814162 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:32.814289 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:32.818949 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:32.818966 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:32.818978 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:32.819012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:32.822141 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:32.822836 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:32.822842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:32.822847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:32.823074 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:32.823517 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:32.823522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:32.823525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:32.824063 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:32.824355 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:32.824794 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 06:23:32.825190 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":403,"threads_started":1}
I20260811 06:23:32.833304 11808 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } }
I20260811 06:23:32.833663 11808 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:32.833772 11808 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00b0cea04984eb2816531c7b82c55e5, State: Initialized, Role: FOLLOWER
I20260811 06:23:32.834280 11808 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } }
I20260811 06:23:32.834520 11808 raft_consensus.cc:388] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:32.834664 11808 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:32.834833 11808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:32.836942 11808 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } }
I20260811 06:23:32.837400 11808 leader_election.cc:302] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d00b0cea04984eb2816531c7b82c55e5; no voters:
I20260811 06:23:32.837953 11808 leader_election.cc:288] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:32.838079 11813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:32.838763 11813 raft_consensus.cc:686] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [term 1 LEADER]: Becoming Leader. State: Replica: d00b0cea04984eb2816531c7b82c55e5, State: Running, Role: LEADER
I20260811 06:23:32.839247 11813 consensus_queue.cc:237] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } }
I20260811 06:23:32.840226 11808 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:32.842507 11814 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d00b0cea04984eb2816531c7b82c55e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } } }
I20260811 06:23:32.842855 11814 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:32.843839 11822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:32.843638 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d00b0cea04984eb2816531c7b82c55e5. Latest consensus state: current_term: 1 leader_uuid: "d00b0cea04984eb2816531c7b82c55e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00b0cea04984eb2816531c7b82c55e5" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38791 } } }
I20260811 06:23:32.844045 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:32.849704 11822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:32.857990 11822 catalog_manager.cc:1347] Generated new cluster ID: 0ba767735c5246f89e34dc7a0714357a
I20260811 06:23:32.858173 11822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:32.872792 11822 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:32.874545 11822 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:32.888309 11822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: Generated new TSK 0
I20260811 06:23:32.889281 11822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:32.943127 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:33.271097 11836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:33.271404 11836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:33.271597 11836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:33.278692 11836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:33.278796 11836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:33.278961 11836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:33.288365 11836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:33.289600 11836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:33.290860 11836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:33.303459 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:33.303491 11842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:33.303390 11845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:33.304462 11836 server_base.cc:1034] running on GCE node
I20260811 06:23:33.304980 11836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:33.305778 11836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:33.307005 11836 hybrid_clock.cc:648] HybridClock initialized: now 1786429413306967 us; error 45 us; skew 500 ppm
I20260811 06:23:33.307364 11836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:33 dist-test-slave-07c2 krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429413, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:33.310554 11836 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:33.313051 11836 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:33.313179 11836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:33.313542 11836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:33.316509 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "0aa6dae42c964252be48e9e768f71553"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "6366254241b00ccdcdbe45790191b14b"
server_key_iv: "7ee78cfce46eddecd31954fe870bf7fd"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.317250 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "0aa6dae42c964252be48e9e768f71553"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "6366254241b00ccdcdbe45790191b14b"
server_key_iv: "7ee78cfce46eddecd31954fe870bf7fd"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.321972 11836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:23:33.325202 11853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.326534 11836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:23:33.326699 11836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "0aa6dae42c964252be48e9e768f71553"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "6366254241b00ccdcdbe45790191b14b"
server_key_iv: "7ee78cfce46eddecd31954fe870bf7fd"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.326897 11836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:33.340867 11836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:33.345144 11836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:33.345435 11836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:33.346371 11836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:33.347662 11836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:33.347783 11836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.347903 11836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:33.347993 11836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.380285 11836 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:44941
I20260811 06:23:33.380374 11974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:44941 every 8 connection(s)
I20260811 06:23:33.381217 11836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:33.384049 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11836
I20260811 06:23:33.384245 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:33.385125 7186 external_mini_cluster.cc:1428] Setting key 494c0f686b9a26e7e7946f532bbb9b61
Aug 11 06:23:33 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429413, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:33.401261 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:33.383487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:55291 (local address 127.7.4.190:38791)
0811 06:23:33.383951 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:33.383955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:33.385003 (+ 1048us) server_negotiation.cc:407] Connection header received
0811 06:23:33.387063 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:33.387069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:33.387160 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:33.387277 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:33.388996 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.389711 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.391348 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.391741 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.391874 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:33.395205 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:33.395210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:33.395213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:33.395242 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:33.398266 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:33.398970 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:33.398976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:33.398981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:33.399102 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:33.399605 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:33.399610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:33.399612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:33.399866 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:33.400022 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:33.400668 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 06:23:33.400881 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":265,"threads_started":1}
I20260811 06:23:33.402141 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:33.384085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:38791 (local address 127.7.4.129:55291)
0811 06:23:33.384860 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:33.384896 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:33.386796 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:33.387535 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:33.387542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:33.388187 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:33.388833 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:33.388850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.389867 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.389872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:33.391182 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:33.391194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.391964 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.391971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:33.392179 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:33.392831 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:33.392856 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:33.394974 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:33.398401 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:33.398409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:33.398423 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:33.398843 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:33.399220 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:33.399225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:33.399229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:33.399478 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:33.399996 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:33.400003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:33.400308 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 06:23:33.400553 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 06:23:33.401056 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":331,"threads_started":1}
I20260811 06:23:33.403816 11975 heartbeater.cc:347] Connected to a master server at 127.7.4.190:38791
I20260811 06:23:33.404106 11975 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:33.404740 11975 heartbeater.cc:511] Master 127.7.4.190:38791 requested a full tablet report, sending...
I20260811 06:23:33.406728 11771 ts_manager.cc:194] Registered new tserver with Master: 0aa6dae42c964252be48e9e768f71553 (127.7.4.129:44941)
I20260811 06:23:33.408140 11771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:55291
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:33.438187 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:33.770608 11982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:33.770900 11982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:33.771083 11982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:33.778363 11982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:33.778510 11982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:33.778687 11982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:33.788308 11982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:33.789561 11982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:33.790870 11982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:33.803089 11989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:33.803660 11988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:33.803166 11991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:33.804718 11982 server_base.cc:1034] running on GCE node
I20260811 06:23:33.805354 11982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:33.806237 11982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:33.807447 11982 hybrid_clock.cc:648] HybridClock initialized: now 1786429413807402 us; error 39 us; skew 500 ppm
I20260811 06:23:33.807821 11982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:33 dist-test-slave-07c2 krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429413, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:33.814692 11982 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:33.815865 11982 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:33.816006 11982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:33.816350 11982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:33.819196 11982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "40778ad374f844eab290b733f5fac8ae"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "1841bc2a7b157e8f7ff573288f338611"
server_key_iv: "adbe68fd4499a613c3dde3382ae0e2ec"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.820047 11982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "40778ad374f844eab290b733f5fac8ae"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "1841bc2a7b157e8f7ff573288f338611"
server_key_iv: "adbe68fd4499a613c3dde3382ae0e2ec"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.824823 11982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:23:33.828051 11999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.829296 11982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:33.829486 11982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "40778ad374f844eab290b733f5fac8ae"
format_stamp: "Formatted at 2026-08-11 06:23:33 on dist-test-slave-07c2"
server_key: "1841bc2a7b157e8f7ff573288f338611"
server_key_iv: "adbe68fd4499a613c3dde3382ae0e2ec"
server_key_version: "encryptionkey@0"
I20260811 06:23:33.829706 11982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:33.854048 11982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:33.858312 11982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:33.858589 11982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:33.859483 11982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:33.860726 11982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:33.860814 11982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.860930 11982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:33.861014 11982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:33.894956 11982 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34535
I20260811 06:23:33.895036 12123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34535 every 8 connection(s)
I20260811 06:23:33.895845 11982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:33.899205 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 11982
I20260811 06:23:33.899443 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:33.899999 7186 external_mini_cluster.cc:1428] Setting key 326b9600513f54a555df5902a519ac3b
Aug 11 06:23:33 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429413, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:33.915237 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:33.898166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:57473 (local address 127.7.4.190:38791)
0811 06:23:33.898353 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:33.898357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:33.899692 (+ 1335us) server_negotiation.cc:407] Connection header received
0811 06:23:33.901694 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:33.901699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:33.901792 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:33.901901 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:33.903331 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.904017 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.905538 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.905982 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.906119 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:33.909539 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:33.909543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:33.909546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:33.909576 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:33.912266 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:33.912989 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:33.912996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:33.912999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:33.913139 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:33.913562 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:33.913568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:33.913572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:33.913835 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:33.913994 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:33.914633 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 06:23:33.914880 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:23:33.916162 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:33.898673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:38791 (local address 127.7.4.130:57473)
0811 06:23:33.899516 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:33.899562 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:33.901421 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:33.902123 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:33.902131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:33.902593 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:33.903126 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:33.903142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.904206 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.904210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:33.905394 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:33.905405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:33.906217 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:33.906224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:33.906440 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:33.907106 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:33.907129 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:33.909315 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:33.912423 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:33.912440 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:33.912453 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:33.912869 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:33.913252 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:33.913257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:33.913260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:33.913442 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:33.913968 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:33.913975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:33.914282 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 06:23:33.914541 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:23:33.915042 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":374,"threads_started":1}
I20260811 06:23:33.917727 12124 heartbeater.cc:347] Connected to a master server at 127.7.4.190:38791
I20260811 06:23:33.918033 12124 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:33.918675 12124 heartbeater.cc:511] Master 127.7.4.190:38791 requested a full tablet report, sending...
I20260811 06:23:33.920126 11771 ts_manager.cc:194] Registered new tserver with Master: 40778ad374f844eab290b733f5fac8ae (127.7.4.130:34535)
I20260811 06:23:33.920912 11771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:57473
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:33.952416 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:34.278422 12129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:34.278676 12129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:34.278779 12129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:34.285722 12129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:34.285810 12129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:34.285915 12129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:34.295137 12129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:38791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:34.296299 12129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:34.297493 12129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:34.309837 12137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:34.309911 12135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:34.309909 12139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:34.311059 12129 server_base.cc:1034] running on GCE node
I20260811 06:23:34.311671 12129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:34.312477 12129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:34.313675 12129 hybrid_clock.cc:648] HybridClock initialized: now 1786429414313626 us; error 38 us; skew 500 ppm
I20260811 06:23:34.314046 12129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:34 dist-test-slave-07c2 krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:34.318007 12129 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:34.319180 12129 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:34.319317 12129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:34.319653 12129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:34.322505 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "93c12e82c8cc4b7492b42cbb89e5f26a"
format_stamp: "Formatted at 2026-08-11 06:23:34 on dist-test-slave-07c2"
server_key: "3da8c98b514ff98b69549a25ccd53515"
server_key_iv: "8e9c351357523900f047e06d4ec90ff2"
server_key_version: "encryptionkey@0"
I20260811 06:23:34.323376 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "93c12e82c8cc4b7492b42cbb89e5f26a"
format_stamp: "Formatted at 2026-08-11 06:23:34 on dist-test-slave-07c2"
server_key: "3da8c98b514ff98b69549a25ccd53515"
server_key_iv: "8e9c351357523900f047e06d4ec90ff2"
server_key_version: "encryptionkey@0"
I20260811 06:23:34.328260 12129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:23:34.331707 12145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:34.333055 12129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:23:34.333225 12129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "93c12e82c8cc4b7492b42cbb89e5f26a"
format_stamp: "Formatted at 2026-08-11 06:23:34 on dist-test-slave-07c2"
server_key: "3da8c98b514ff98b69549a25ccd53515"
server_key_iv: "8e9c351357523900f047e06d4ec90ff2"
server_key_version: "encryptionkey@0"
I20260811 06:23:34.333431 12129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:34.353866 12129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:34.358278 12129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:34.358556 12129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:34.359586 12129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:34.361011 12129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:34.361123 12129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:34.361255 12129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:34.361340 12129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:34.394518 12129 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:40311
I20260811 06:23:34.394580 12268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:40311 every 8 connection(s)
I20260811 06:23:34.395503 12129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:34.403018 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12129
I20260811 06:23:34.403191 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:34.403750 7186 external_mini_cluster.cc:1428] Setting key 1782e3a17b65d3a1437eb00fe6ff1f3f
Aug 11 06:23:34 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429414, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:34.411067 11975 heartbeater.cc:503] Master 127.7.4.190:38791 was elected leader, sending a full tablet report...
I20260811 06:23:34.415007 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.397952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:49581 (local address 127.7.4.190:38791)
0811 06:23:34.398157 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.398163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.399333 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 06:23:34.400950 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.400955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.401055 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.401138 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:34.402407 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.403413 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.405373 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.405991 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.406116 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.409551 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:34.409556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:34.409559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:34.409590 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:34.412456 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.413155 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.413158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.413160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:34.413288 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.413639 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.413642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.413644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:34.413904 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:34.414026 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.414610 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.414771 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:23:34.415927 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.398493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:38791 (local address 127.7.4.131:49581)
0811 06:23:34.399159 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.399193 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.400726 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.401312 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.401318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.401781 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:34.402271 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.402286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.403538 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.403541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.405205 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.405219 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.406226 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.406234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.406509 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.407232 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:34.407253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:34.409378 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:34.412610 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:34.412618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:34.412631 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:34.413050 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:34.413383 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:34.413386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:34.413388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:34.413549 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:34.414021 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:34.414026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:34.414313 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.414525 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.414872 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":292,"threads_started":1}
I20260811 06:23:34.417286 12269 heartbeater.cc:347] Connected to a master server at 127.7.4.190:38791
I20260811 06:23:34.417582 12269 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:34.418273 12269 heartbeater.cc:511] Master 127.7.4.190:38791 requested a full tablet report, sending...
I20260811 06:23:34.419685 11771 ts_manager.cc:194] Registered new tserver with Master: 93c12e82c8cc4b7492b42cbb89e5f26a (127.7.4.131:40311)
I20260811 06:23:34.420467 11771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:49581
I20260811 06:23:34.431592 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:23:34.448649 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.436046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.7.4.190:38791)
0811 06:23:34.436236 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.436241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.436795 (+ 554us) server_negotiation.cc:407] Connection header received
0811 06:23:34.437009 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.437014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.437087 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.437190 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:34.438678 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.439469 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.440742 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.441201 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.441336 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.443510 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:34.443520 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:34.443522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:34.443551 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:34.445885 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.446574 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.446582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.446585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:34.446734 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.447118 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.447124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.447127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:34.447510 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:34.447813 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.448123 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.448336 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:23:34.452417 11771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:34.459555 11771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:34.481560 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.471780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.7.4.131:40311)
0811 06:23:34.473479 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.473486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.473519 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:23:34.473612 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.473619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.473831 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.474349 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:34.474877 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.476548 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.478903 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.480352 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.480483 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.480721 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.481016 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.481180 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1526,"thread_start_us":1375,"threads_started":1}
I20260811 06:23:34.481861 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.470917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40311 (local address 127.0.0.1:59516)
0811 06:23:34.472284 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.472301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.472413 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.474041 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.474048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.474096 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:34.474668 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.474680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.476708 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.476714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.478691 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.478716 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.480555 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.480563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.480762 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.480818 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.480962 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.481139 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"mutex_wait_us":134,"spinlock_wait_cycles":896,"thread_start_us":382,"threads_started":1}
I20260811 06:23:34.482221 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.471073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.7.4.129:44941)
0811 06:23:34.472993 (+ 1920us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.473004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.473036 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:23:34.473161 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.473170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.473418 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.473614 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:34.474665 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.476564 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.479325 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.480399 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.480552 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.480793 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.481644 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.481870 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1731,"thread_start_us":1551,"threads_started":1}
I20260811 06:23:34.482481 12287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.470721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45334)
0811 06:23:34.471646 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.471695 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.471848 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.473923 (+ 2075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.473932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.473967 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:34.474478 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.474495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.476744 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.476748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.479112 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.479128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.480947 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.480954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.481102 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.481130 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.481215 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.481356 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":24,"thread_start_us":1284,"threads_started":1}
I20260811 06:23:34.483038 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.472976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.7.4.130:34535)
0811 06:23:34.473777 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.473788 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.474606 (+ 818us) server_negotiation.cc:407] Connection header received
0811 06:23:34.474729 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.474738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.474923 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.475107 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:34.475909 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.477289 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.479807 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.481132 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.481274 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.481476 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.482383 (+ 907us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.482542 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":521,"threads_started":1}
I20260811 06:23:34.483132 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.472454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34535 (local address 127.0.0.1:35862)
0811 06:23:34.474284 (+ 1830us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.474316 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.474453 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.475245 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.475248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.475271 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:34.475702 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.475714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.477441 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.477447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.479634 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.479648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.481709 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.481718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.481938 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.481986 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.482071 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.482210 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1716,"mutex_wait_us":3,"thread_start_us":1528,"threads_started":1}
I20260811 06:23:34.486358 12199 tablet_service.cc:1467] Processing CreateTablet for tablet 748b6a7838ce432c8c2a825062d729dc (DEFAULT_TABLE table=test-table [id=8291db9291614a25b74af7429e42f12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:34.486562 12051 tablet_service.cc:1467] Processing CreateTablet for tablet 748b6a7838ce432c8c2a825062d729dc (DEFAULT_TABLE table=test-table [id=8291db9291614a25b74af7429e42f12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:34.486572 11904 tablet_service.cc:1467] Processing CreateTablet for tablet 748b6a7838ce432c8c2a825062d729dc (DEFAULT_TABLE table=test-table [id=8291db9291614a25b74af7429e42f12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:34.487707 12199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748b6a7838ce432c8c2a825062d729dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:34.487947 11904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748b6a7838ce432c8c2a825062d729dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:34.487946 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748b6a7838ce432c8c2a825062d729dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:34.498682 12293 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Bootstrap starting.
I20260811 06:23:34.502375 12296 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Bootstrap starting.
I20260811 06:23:34.502632 12293 tablet_bootstrap.cc:654] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:34.503996 12295 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Bootstrap starting.
I20260811 06:23:34.504062 12293 log.cc:824] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:34.506587 12296 tablet_bootstrap.cc:654] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:34.507102 12295 tablet_bootstrap.cc:654] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:34.507876 12293 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: No bootstrap required, opened a new log
I20260811 06:23:34.507833 12296 log.cc:824] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:34.508136 12293 ts_tablet_manager.cc:1397] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 06:23:34.508138 12295 log.cc:824] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:34.511057 12295 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: No bootstrap required, opened a new log
I20260811 06:23:34.511368 12295 ts_tablet_manager.cc:1397] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 06:23:34.517759 12296 tablet_bootstrap.cc:492] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: No bootstrap required, opened a new log
I20260811 06:23:34.518020 12296 ts_tablet_manager.cc:1397] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260811 06:23:34.518407 12295 raft_consensus.cc:348] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.518759 12295 raft_consensus.cc:374] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:34.518890 12295 raft_consensus.cc:729] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aa6dae42c964252be48e9e768f71553, State: Initialized, Role: FOLLOWER
I20260811 06:23:34.519500 12295 consensus_queue.cc:260] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.520368 12293 raft_consensus.cc:348] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.520828 12293 raft_consensus.cc:374] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:34.520965 12293 raft_consensus.cc:729] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c12e82c8cc4b7492b42cbb89e5f26a, State: Initialized, Role: FOLLOWER
I20260811 06:23:34.521510 12293 consensus_queue.cc:260] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.522763 12269 heartbeater.cc:503] Master 127.7.4.190:38791 was elected leader, sending a full tablet report...
I20260811 06:23:34.524610 12293 ts_tablet_manager.cc:1428] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260811 06:23:34.526185 12295 ts_tablet_manager.cc:1428] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260811 06:23:34.528239 12296 raft_consensus.cc:348] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.528668 12296 raft_consensus.cc:374] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:34.528784 12296 raft_consensus.cc:729] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40778ad374f844eab290b733f5fac8ae, State: Initialized, Role: FOLLOWER
I20260811 06:23:34.529359 12296 consensus_queue.cc:260] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.530809 12124 heartbeater.cc:503] Master 127.7.4.190:38791 was elected leader, sending a full tablet report...
I20260811 06:23:34.531142 12296 ts_tablet_manager.cc:1428] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260811 06:23:34.633620 11976 tablet.cc:2369] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:34.646974 12271 tablet.cc:2369] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:34.647627 12125 tablet.cc:2369] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:34.831859 12306 raft_consensus.cc:482] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:34.832195 12306 raft_consensus.cc:504] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.834380 12306 leader_election.cc:288] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0aa6dae42c964252be48e9e768f71553 (127.7.4.129:44941), 93c12e82c8cc4b7492b42cbb89e5f26a (127.7.4.131:40311)
I20260811 06:23:34.843072 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.834945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:42327 (local address 127.7.4.131:40311)
0811 06:23:34.835148 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.835153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.835623 (+ 470us) server_negotiation.cc:407] Connection header received
0811 06:23:34.835972 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.835981 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.836063 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.836175 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:34.836822 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.837980 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.840659 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.841873 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.842001 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.842103 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.842662 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.842813 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:34.843079 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.834898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:51823 (local address 127.7.4.129:44941)
0811 06:23:34.835137 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.835146 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.836040 (+ 894us) server_negotiation.cc:407] Connection header received
0811 06:23:34.836242 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.836246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.836301 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.836401 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:34.837102 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.838138 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.840659 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.841873 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.842001 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.842103 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.842662 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.842813 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:23:34.843348 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.834628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40311 (local address 127.7.4.130:42327)
0811 06:23:34.835512 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.835533 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.835771 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.836195 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.836199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.836236 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:34.836668 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.836679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.838094 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.838098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.840496 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.840504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.842166 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.842171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.842465 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.842526 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.842622 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.842806 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":122,"thread_start_us":510,"threads_started":1}
I20260811 06:23:34.843372 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.834750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.7.4.130:51823)
0811 06:23:34.835950 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:34.835967 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:34.836105 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:34.836422 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:34.836426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:34.836471 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:34.836913 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.836922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.838284 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.838289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.840489 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:34.840503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.842162 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.842169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:34.842472 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.842531 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 06:23:34.842622 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:23:34.842806 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":188,"thread_start_us":857,"threads_started":1}
I20260811 06:23:34.844594 12221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748b6a7838ce432c8c2a825062d729dc" candidate_uuid: "40778ad374f844eab290b733f5fac8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" is_pre_election: true
I20260811 06:23:34.844594 11928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748b6a7838ce432c8c2a825062d729dc" candidate_uuid: "40778ad374f844eab290b733f5fac8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aa6dae42c964252be48e9e768f71553" is_pre_election: true
I20260811 06:23:34.845155 11928 raft_consensus.cc:2454] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40778ad374f844eab290b733f5fac8ae in term 0.
I20260811 06:23:34.845155 12221 raft_consensus.cc:2454] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40778ad374f844eab290b733f5fac8ae in term 0.
I20260811 06:23:34.845899 12001 leader_election.cc:302] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40778ad374f844eab290b733f5fac8ae, 93c12e82c8cc4b7492b42cbb89e5f26a; no voters:
I20260811 06:23:34.846352 12306 raft_consensus.cc:2781] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:34.846535 12306 raft_consensus.cc:482] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:34.846649 12306 raft_consensus.cc:3037] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:34.848899 12306 raft_consensus.cc:504] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.849459 12306 leader_election.cc:288] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [CANDIDATE]: Term 1 election: Requested vote from peers 0aa6dae42c964252be48e9e768f71553 (127.7.4.129:44941), 93c12e82c8cc4b7492b42cbb89e5f26a (127.7.4.131:40311)
I20260811 06:23:34.849893 11928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748b6a7838ce432c8c2a825062d729dc" candidate_uuid: "40778ad374f844eab290b733f5fac8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0aa6dae42c964252be48e9e768f71553"
I20260811 06:23:34.849921 12221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748b6a7838ce432c8c2a825062d729dc" candidate_uuid: "40778ad374f844eab290b733f5fac8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a"
I20260811 06:23:34.850071 11928 raft_consensus.cc:3037] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:34.850114 12221 raft_consensus.cc:3037] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:34.852324 11928 raft_consensus.cc:2454] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40778ad374f844eab290b733f5fac8ae in term 1.
I20260811 06:23:34.852434 12221 raft_consensus.cc:2454] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40778ad374f844eab290b733f5fac8ae in term 1.
I20260811 06:23:34.852795 12003 leader_election.cc:302] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aa6dae42c964252be48e9e768f71553, 40778ad374f844eab290b733f5fac8ae; no voters:
I20260811 06:23:34.853129 12306 raft_consensus.cc:2781] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:34.853559 12306 raft_consensus.cc:686] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 1 LEADER]: Becoming Leader. State: Replica: 40778ad374f844eab290b733f5fac8ae, State: Running, Role: LEADER
I20260811 06:23:34.854061 12306 consensus_queue.cc:237] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } }
I20260811 06:23:34.858232 11771 catalog_manager.cc:5495] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 40778ad374f844eab290b733f5fac8ae (127.7.4.130). New cstate: current_term: 1 leader_uuid: "40778ad374f844eab290b733f5fac8ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40778ad374f844eab290b733f5fac8ae" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34535 } health_report { overall_health: HEALTHY } } }
I20260811 06:23:34.933727 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.928223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.7.4.130:34535)
0811 06:23:34.928413 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.928421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.928442 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:23:34.928720 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.928725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.928810 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.928958 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:34.929339 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.930094 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.931564 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.931966 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.932111 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.932579 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:34.933126 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:34.933285 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:34.933343 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 06:23:34.933422 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:23:34.941740 11928 raft_consensus.cc:1261] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 1 FOLLOWER]: Refusing update from remote peer 40778ad374f844eab290b733f5fac8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:34.941715 12221 raft_consensus.cc:1261] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 1 FOLLOWER]: Refusing update from remote peer 40778ad374f844eab290b733f5fac8ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:34.942723 12310 consensus_queue.cc:1035] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c12e82c8cc4b7492b42cbb89e5f26a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:34.943071 12306 consensus_queue.cc:1035] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0aa6dae42c964252be48e9e768f71553" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:34.969041 11771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:34.969310 11771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:34.972749 11771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d00b0cea04984eb2816531c7b82c55e5: Sending DeleteTablet for 3 replicas of tablet 748b6a7838ce432c8c2a825062d729dc
I20260811 06:23:34.973831 12199 tablet_service.cc:1514] Processing DeleteTablet for tablet 748b6a7838ce432c8c2a825062d729dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:59516
I20260811 06:23:34.974193 11904 tablet_service.cc:1514] Processing DeleteTablet for tablet 748b6a7838ce432c8c2a825062d729dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:45334
I20260811 06:23:34.974831 12051 tablet_service.cc:1514] Processing DeleteTablet for tablet 748b6a7838ce432c8c2a825062d729dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35862
I20260811 06:23:34.976058 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11836
I20260811 06:23:34.987802 12324 tablet_replica.cc:330] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: stopping tablet replica
I20260811 06:23:34.988878 12324 raft_consensus.cc:2229] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:34.990059 12324 raft_consensus.cc:2258] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 06:23:34 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:34.992878 12323 tablet_replica.cc:330] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: stopping tablet replica
I20260811 06:23:34.993078 12324 ts_tablet_manager.cc:1905] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:34.993563 12323 raft_consensus.cc:2229] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:34.994102 12323 raft_consensus.cc:2258] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:34.996117 12325 tablet_replica.cc:330] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: stopping tablet replica
I20260811 06:23:34.996385 12323 ts_tablet_manager.cc:1905] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:34.996570 12325 raft_consensus.cc:2229] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:34.997355 12325 raft_consensus.cc:2258] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:35.002301 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:34.977268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.7.4.129:44941)
0811 06:23:34.977493 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:34.977502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:34.979147 (+ 1645us) server_negotiation.cc:407] Connection header received
0811 06:23:34.979473 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:34.979482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:34.979601 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:34.979714 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:34.981906 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.983436 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.985852 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:34.986843 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:34.986969 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:34.993176 (+ 6207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:34.993193 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:34.993209 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:34.993251 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:34.997592 (+ 4341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.998428 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.998438 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.998442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:34.998626 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:34.999093 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:34.999098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:34.999102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:35.000097 (+ 995us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:35.000438 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:35.001608 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0811 06:23:35.001855 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:23:35.003208 11970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:35.003239 12324 ts_tablet_manager.cc:1918] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:35.004244 12324 log.cc:1196] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/748b6a7838ce432c8c2a825062d729dc
I20260811 06:23:35.005631 12325 ts_tablet_manager.cc:1905] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:35.006721 12323 ts_tablet_manager.cc:1918] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:35.006968 12323 log.cc:1196] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/748b6a7838ce432c8c2a825062d729dc
I20260811 06:23:35.007606 12323 ts_tablet_manager.cc:1939] T 748b6a7838ce432c8c2a825062d729dc P 93c12e82c8cc4b7492b42cbb89e5f26a: Deleting consensus metadata
I20260811 06:23:35.009665 11755 catalog_manager.cc:4841] TS 93c12e82c8cc4b7492b42cbb89e5f26a (127.7.4.131:40311): tablet 748b6a7838ce432c8c2a825062d729dc (table test-table [id=8291db9291614a25b74af7429e42f12c]) successfully deleted
I20260811 06:23:35.014492 12325 ts_tablet_manager.cc:1918] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:35.014643 12325 log.cc:1196] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/748b6a7838ce432c8c2a825062d729dc
I20260811 06:23:35.015189 12325 ts_tablet_manager.cc:1939] T 748b6a7838ce432c8c2a825062d729dc P 40778ad374f844eab290b733f5fac8ae: Deleting consensus metadata
I20260811 06:23:35.016880 11757 catalog_manager.cc:4841] TS 40778ad374f844eab290b733f5fac8ae (127.7.4.130:34535): tablet 748b6a7838ce432c8c2a825062d729dc (table test-table [id=8291db9291614a25b74af7429e42f12c]) successfully deleted
I20260811 06:23:35.101222 12324 ts_tablet_manager.cc:1939] T 748b6a7838ce432c8c2a825062d729dc P 0aa6dae42c964252be48e9e768f71553: Deleting consensus metadata
I20260811 06:23:35.101311 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11836
I20260811 06:23:35.125156 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11982
W20260811 06:23:35.125128 11757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.4.129:44941 (error 108)
I20260811 06:23:35.128935 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.126797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45346)
0811 06:23:35.128483 (+ 1686us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.128688 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1564,"thread_start_us":1465,"threads_started":1}
W20260811 06:23:35.129289 11757 catalog_manager.cc:4584] TS 0aa6dae42c964252be48e9e768f71553 (127.7.4.129:44941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 748b6a7838ce432c8c2a825062d729dc: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Aug 11 06:23:35 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:35.142040 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.126384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.7.4.130:34535)
0811 06:23:35.126575 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:35.126580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:35.126597 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:23:35.126657 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:35.126661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:35.126721 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:35.126846 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:35.128185 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:35.129368 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:35.130681 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:35.131110 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:35.131279 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:35.135695 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:35.135711 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:35.135725 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:35.135765 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:35.138906 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:35.139829 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:35.139838 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:35.139841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:35.139986 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:35.140400 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:35.140404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:35.140408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:35.140769 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:35.141136 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:35.141429 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 06:23:35.141662 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:23:35.142748 12119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:35.182862 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.182426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45354)
0811 06:23:35.182634 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.182728 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 06:23:35.183954 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.183554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45364)
0811 06:23:35.183727 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.183817 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:23:35.244273 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.243801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45370)
0811 06:23:35.244034 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.244119 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:23:35.245330 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.244969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45372)
0811 06:23:35.245123 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.245190 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:23:35.255800 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11982
I20260811 06:23:35.279659 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12129
Aug 11 06:23:35 dist-test-slave-07c2 krb5kdc[11723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:35.294344 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.280430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.7.4.131:40311)
0811 06:23:35.280596 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:35.280600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:35.280623 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:23:35.280793 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:35.280797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:35.280872 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:35.280957 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:35.282182 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:35.283077 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:35.284477 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:35.284924 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:35.285024 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:35.289277 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:35.289290 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:35.289305 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:35.289337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:35.292002 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:35.292594 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:35.292602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:35.292604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:35.292786 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:35.293106 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:35.293109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:35.293111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:35.293456 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:35.293675 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:35.293925 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 06:23:35.294080 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:23:35.294950 12263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:35.325466 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.325012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45382)
0811 06:23:35.325221 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.325315 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:23:35.326583 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.326168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45390)
0811 06:23:35.326382 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.326461 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 06:23:35.385735 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12129
I20260811 06:23:35.406565 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 11737
I20260811 06:23:35.407809 11802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:35.516991 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:35.516386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:44941 (local address 127.0.0.1:45400)
0811 06:23:35.516631 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:35.516801 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 06:23:35.517201 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 11737
2026-08-11T06:23:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4507 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T06:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:35Z Disabled control of system clock
I20260811 06:23:35.599715 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44405
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:44405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:23:35.918638 12336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:35.918897 12336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:35.918979 12336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:35.925719 12336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:35.925807 12336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:35.925843 12336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:35.925908 12336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:35.925943 12336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:35.934675 12336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:44405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:35.935782 12336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:35.937543 12336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:35.949149 12341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:35.949231 12342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:35.949251 12344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:35.950857 12336 server_base.cc:1034] running on GCE node
I20260811 06:23:35.951630 12336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:35.952886 12336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:35.954139 12336 hybrid_clock.cc:648] HybridClock initialized: now 1786429415954087 us; error 43 us; skew 500 ppm
I20260811 06:23:35.954468 12336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:35.956667 12336 webserver.cc:460] Webserver started at http://127.7.4.190:42689/ using document root <none> and password file <none>
I20260811 06:23:35.957321 12336 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:35.957458 12336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:35.957792 12336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:35.960604 12336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "4f471cacc6f34f03bf7f12ae0dea3144"
format_stamp: "Formatted at 2026-08-11 06:23:35 on dist-test-slave-07c2"
server_key: "9e582c29bceb4c77a44d5b14bf3d8318"
server_key_iv: "744bc656c5a13c7d9156bdac5536dd20"
server_key_version: "encryptionkey@0"
I20260811 06:23:35.961429 12336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "4f471cacc6f34f03bf7f12ae0dea3144"
format_stamp: "Formatted at 2026-08-11 06:23:35 on dist-test-slave-07c2"
server_key: "9e582c29bceb4c77a44d5b14bf3d8318"
server_key_iv: "744bc656c5a13c7d9156bdac5536dd20"
server_key_version: "encryptionkey@0"
I20260811 06:23:35.966395 12336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 06:23:35.969633 12351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:35.970851 12336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:35.971028 12336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "4f471cacc6f34f03bf7f12ae0dea3144"
format_stamp: "Formatted at 2026-08-11 06:23:35 on dist-test-slave-07c2"
server_key: "9e582c29bceb4c77a44d5b14bf3d8318"
server_key_iv: "744bc656c5a13c7d9156bdac5536dd20"
server_key_version: "encryptionkey@0"
I20260811 06:23:35.971243 12336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:35.976724 12336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:35.977433 12336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:35.977743 12336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:35.995658 12336 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:44405
I20260811 06:23:35.995682 12407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:44405 every 8 connection(s)
I20260811 06:23:35.996917 12336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:35.997629 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12336
I20260811 06:23:35.997879 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:35.998616 7186 external_mini_cluster.cc:1428] Setting key b472060396c1665d8e67713e9517a932
I20260811 06:23:36.001075 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:36.006778 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:36.000595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.7.4.190:44405)
0811 06:23:36.001623 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:36.001633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:36.001671 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:23:36.003172 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:36.003192 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:36.003646 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:36.004025 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:36.005112 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:36.005121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:36.005133 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:36.005151 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:36.005227 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:36.005247 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:36.005507 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:36.005656 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 06:23:36.005902 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":382,"threads_started":1}
I20260811 06:23:36.012647 12408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: Bootstrap starting.
I20260811 06:23:36.015833 12408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:36.016973 12408 log.cc:824] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:36.019623 12408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: No bootstrap required, opened a new log
I20260811 06:23:36.027524 12408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } }
I20260811 06:23:36.027846 12408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:36.027918 12408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f471cacc6f34f03bf7f12ae0dea3144, State: Initialized, Role: FOLLOWER
I20260811 06:23:36.028373 12408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } }
I20260811 06:23:36.028595 12408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:36.028695 12408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:36.028824 12408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:36.030798 12408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } }
I20260811 06:23:36.031217 12408 leader_election.cc:302] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f471cacc6f34f03bf7f12ae0dea3144; no voters:
I20260811 06:23:36.031729 12408 leader_election.cc:288] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:36.031876 12414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:36.032629 12414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [term 1 LEADER]: Becoming Leader. State: Replica: 4f471cacc6f34f03bf7f12ae0dea3144, State: Running, Role: LEADER
I20260811 06:23:36.033209 12414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } }
I20260811 06:23:36.034013 12408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:36.036371 12416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f471cacc6f34f03bf7f12ae0dea3144. Latest consensus state: current_term: 1 leader_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } } }
I20260811 06:23:36.036741 12416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:36.036404 12415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f471cacc6f34f03bf7f12ae0dea3144" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44405 } } }
I20260811 06:23:36.036870 12415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:36.038252 12421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:36.043423 12421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:36.051571 12421 catalog_manager.cc:1347] Generated new cluster ID: d3d88bc9c85749f8bb0b9deb29059a8e
I20260811 06:23:36.051679 12421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:36.072340 12421 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:36.072695 12421 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:36.081425 12421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: Generated new TSK 0
I20260811 06:23:36.082064 12421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:23:36.091974 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--builtin_ntp_servers=127.7.4.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:23:36.420254 12435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:36.420500 12435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:36.420629 12435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:36.427600 12435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:36.427691 12435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:36.427855 12435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:36.436780 12435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:36.437927 12435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:36.439751 12435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:36.452709 12442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:36.455413 12444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:36.452696 12441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:36.456346 12435 server_base.cc:1034] running on GCE node
I20260811 06:23:36.456912 12435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:36.457780 12435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:36.459018 12435 hybrid_clock.cc:648] HybridClock initialized: now 1786429416458985 us; error 42 us; skew 500 ppm
I20260811 06:23:36.459390 12435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:36.461508 12435 webserver.cc:460] Webserver started at http://127.7.4.129:38147/ using document root <none> and password file <none>
I20260811 06:23:36.462200 12435 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:36.462325 12435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:36.462726 12435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:36.465617 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "02b3c4f03edd41be92e16681a1095412"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "ae3fae61b04f989905776dd61277720b"
server_key_iv: "e91b185cc680162b0f4c38740281e4b4"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.466413 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "02b3c4f03edd41be92e16681a1095412"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "ae3fae61b04f989905776dd61277720b"
server_key_iv: "e91b185cc680162b0f4c38740281e4b4"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.471359 12435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:36.474622 12451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.475935 12435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:36.476114 12435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "02b3c4f03edd41be92e16681a1095412"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "ae3fae61b04f989905776dd61277720b"
server_key_iv: "e91b185cc680162b0f4c38740281e4b4"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.476332 12435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:36.481700 12435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:36.482456 12435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:36.482735 12435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:36.483718 12435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:36.485013 12435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:36.485136 12435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.485265 12435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:36.485347 12435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.518355 12435 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39483
I20260811 06:23:36.518455 12570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39483 every 8 connection(s)
I20260811 06:23:36.519722 12435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:36.524544 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12435
I20260811 06:23:36.524775 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:36.525415 7186 external_mini_cluster.cc:1428] Setting key 8415844b9a65b2b32f5d47fc385d5821
I20260811 06:23:36.528062 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:36.522247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44215 (local address 127.7.4.190:44405)
0811 06:23:36.522707 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:36.522714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:36.523829 (+ 1115us) server_negotiation.cc:407] Connection header received
0811 06:23:36.525578 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:36.525583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:36.525656 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:36.525778 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:36.527154 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:36.527163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:36.527170 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:36.527194 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:36.527247 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:36.527266 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:36.527417 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:36.527681 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:23:36.527746 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":205,"threads_started":1}
I20260811 06:23:36.528587 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--builtin_ntp_servers=127.7.4.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:23:36.528924 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:36.522807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44405 (local address 127.7.4.129:44215)
0811 06:23:36.523694 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:36.523727 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:36.525328 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:36.525998 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:36.526005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:36.526087 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:36.526793 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:36.526797 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:36.526843 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:36.526943 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:36.527420 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:36.527432 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:36.527440 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:23:36.527699 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:23:36.527883 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":263,"threads_started":1}
I20260811 06:23:36.530364 12571 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44405
I20260811 06:23:36.530651 12571 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:36.531190 12571 heartbeater.cc:511] Master 127.7.4.190:44405 requested a full tablet report, sending...
I20260811 06:23:36.533174 12370 ts_manager.cc:194] Registered new tserver with Master: 02b3c4f03edd41be92e16681a1095412 (127.7.4.129:39483)
W20260811 06:23:36.862072 12576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:36.862313 12576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:36.862452 12576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:36.869426 12576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:36.869514 12576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:36.869717 12576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:36.878641 12576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:36.879956 12576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:36.881805 12576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:36.894586 12582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:36.894658 12583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:36.894672 12585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:36.900619 12576 server_base.cc:1034] running on GCE node
I20260811 06:23:36.901113 12576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:36.901929 12576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:36.903142 12576 hybrid_clock.cc:648] HybridClock initialized: now 1786429416903077 us; error 39 us; skew 500 ppm
I20260811 06:23:36.903481 12576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:36.905714 12576 webserver.cc:460] Webserver started at http://127.7.4.130:39561/ using document root <none> and password file <none>
I20260811 06:23:36.906379 12576 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:36.906495 12576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:36.906841 12576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:36.909736 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "6f65c058b8fe4596a64659e6a156bcac"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "2c1bc2034da8eb9af1324fd2bdb542a5"
server_key_iv: "e18e754b7b71c73333720470324894e3"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.910506 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "6f65c058b8fe4596a64659e6a156bcac"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "2c1bc2034da8eb9af1324fd2bdb542a5"
server_key_iv: "e18e754b7b71c73333720470324894e3"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.915573 12576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:23:36.918823 12592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.920132 12576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:36.920310 12576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "6f65c058b8fe4596a64659e6a156bcac"
format_stamp: "Formatted at 2026-08-11 06:23:36 on dist-test-slave-07c2"
server_key: "2c1bc2034da8eb9af1324fd2bdb542a5"
server_key_iv: "e18e754b7b71c73333720470324894e3"
server_key_version: "encryptionkey@0"
I20260811 06:23:36.920536 12576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:36.925895 12576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:36.926646 12576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:36.926914 12576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:36.927870 12576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:36.929203 12576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:36.929324 12576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.929458 12576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:36.929556 12576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:36.963263 12576 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:35531
I20260811 06:23:36.963397 12715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:35531 every 8 connection(s)
I20260811 06:23:36.964560 12576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:36.972170 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:36.966974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:34581 (local address 127.7.4.190:44405)
0811 06:23:36.967198 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:36.967205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:36.968571 (+ 1366us) server_negotiation.cc:407] Connection header received
0811 06:23:36.970181 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:36.970185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:36.970247 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:36.970334 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:36.971582 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:36.971586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:36.971589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:36.971605 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:36.971637 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:36.971649 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:36.971754 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:36.971915 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:23:36.971964 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:23:36.972878 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12576
I20260811 06:23:36.973079 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:36.973265 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:36.967545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44405 (local address 127.7.4.130:34581)
0811 06:23:36.968420 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:36.968453 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:36.969957 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:36.970519 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:36.970527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:36.970604 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:36.971236 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:36.971238 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:36.971272 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:36.971398 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:36.971746 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:36.971754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:36.971759 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:23:36.971955 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:23:36.972123 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":281,"threads_started":1}
I20260811 06:23:36.973721 7186 external_mini_cluster.cc:1428] Setting key 0631e8296782c1b0db1865f8979f688f
I20260811 06:23:36.974608 12716 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44405
I20260811 06:23:36.974926 12716 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:36.975584 12716 heartbeater.cc:511] Master 127.7.4.190:44405 requested a full tablet report, sending...
I20260811 06:23:36.976768 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--builtin_ntp_servers=127.7.4.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:23:36.977029 12370 ts_manager.cc:194] Registered new tserver with Master: 6f65c058b8fe4596a64659e6a156bcac (127.7.4.130:35531)
W20260811 06:23:37.317687 12720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:37.317945 12720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:37.318051 12720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:37.327672 12720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:37.327768 12720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:37.327880 12720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:37.337301 12720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:37.338433 12720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:37.340601 12720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:37.354350 12726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:37.354269 12729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:37.354319 12725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:37.360384 12720 server_base.cc:1034] running on GCE node
I20260811 06:23:37.360973 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:37.361871 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:37.363096 12720 hybrid_clock.cc:648] HybridClock initialized: now 1786429417363053 us; error 56 us; skew 500 ppm
I20260811 06:23:37.363461 12720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:37.365584 12720 webserver.cc:460] Webserver started at http://127.7.4.131:36995/ using document root <none> and password file <none>
I20260811 06:23:37.366261 12720 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:37.366389 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:37.366724 12720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:37.369614 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "2f9b14275a5d4f6f854911ef70ffce21"
format_stamp: "Formatted at 2026-08-11 06:23:37 on dist-test-slave-07c2"
server_key: "72a34b22d1f9ed342bac0fd38bb86bc3"
server_key_iv: "24da2c9d85e02b55e1ab1632fe785d6a"
server_key_version: "encryptionkey@0"
I20260811 06:23:37.370477 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9b14275a5d4f6f854911ef70ffce21"
format_stamp: "Formatted at 2026-08-11 06:23:37 on dist-test-slave-07c2"
server_key: "72a34b22d1f9ed342bac0fd38bb86bc3"
server_key_iv: "24da2c9d85e02b55e1ab1632fe785d6a"
server_key_version: "encryptionkey@0"
I20260811 06:23:37.375550 12720 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:37.378877 12735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:37.380327 12720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:37.380486 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "2f9b14275a5d4f6f854911ef70ffce21"
format_stamp: "Formatted at 2026-08-11 06:23:37 on dist-test-slave-07c2"
server_key: "72a34b22d1f9ed342bac0fd38bb86bc3"
server_key_iv: "24da2c9d85e02b55e1ab1632fe785d6a"
server_key_version: "encryptionkey@0"
I20260811 06:23:37.380703 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:37.386194 12720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:37.386920 12720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:37.387135 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:37.388046 12720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:37.389472 12720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:37.389564 12720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:37.389642 12720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:37.389698 12720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:37.422811 12720 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:33729
I20260811 06:23:37.422854 12857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:33729 every 8 connection(s)
I20260811 06:23:37.424223 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:37.431033 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12720
I20260811 06:23:37.431268 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:37.431860 7186 external_mini_cluster.cc:1428] Setting key 58896108fbd3c71e018625f9a19241e9
I20260811 06:23:37.432464 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.426787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:52899 (local address 127.7.4.190:44405)
0811 06:23:37.426963 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.426968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.428351 (+ 1383us) server_negotiation.cc:407] Connection header received
0811 06:23:37.429945 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.429949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.430003 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.430091 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.431554 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.431562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.431569 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.431590 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.431634 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.431647 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.431750 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.432102 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.432174 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:23:37.433395 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.427258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44405 (local address 127.7.4.131:52899)
0811 06:23:37.428182 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.428235 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.429722 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.430287 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.430294 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.430376 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.431076 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.431079 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.431124 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.431266 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.431844 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.431852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.431857 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.432165 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.432405 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":276,"threads_started":1}
I20260811 06:23:37.434736 12858 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44405
I20260811 06:23:37.435056 12858 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:37.435678 12858 heartbeater.cc:511] Master 127.7.4.190:44405 requested a full tablet report, sending...
I20260811 06:23:37.436894 12370 ts_manager.cc:194] Registered new tserver with Master: 2f9b14275a5d4f6f854911ef70ffce21 (127.7.4.131:33729)
I20260811 06:23:37.447003 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:23:37.453610 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.451378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.7.4.190:44405)
0811 06:23:37.451586 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.451595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.451965 (+ 370us) server_negotiation.cc:407] Connection header received
0811 06:23:37.452082 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.452086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.452131 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.452206 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.453020 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.453025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.453028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.453042 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.453073 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.453085 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.453191 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.453286 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.453328 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:23:37.457160 12370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:37.463753 12370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:37.481175 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.474730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43600)
0811 06:23:37.476111 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.476157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.476296 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.478360 (+ 2064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.478371 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.478482 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.479403 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.479409 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.479446 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.479553 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.480192 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.480208 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.480217 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.480433 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.480538 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":500,"threads_started":1}
I20260811 06:23:37.482007 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.475018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.7.4.129:39483)
0811 06:23:37.475966 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.475976 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.476278 (+ 302us) server_negotiation.cc:407] Connection header received
0811 06:23:37.476512 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.476524 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.476809 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.476996 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.479802 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.479815 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.479835 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.479873 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.479948 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.479975 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.480126 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.481401 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.481526 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":3128,"threads_started":1}
I20260811 06:23:37.486733 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.476032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:35531 (local address 127.0.0.1:45518)
0811 06:23:37.482375 (+ 6343us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.482400 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.482501 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.485051 (+ 2550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.485056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.485109 (+ 53us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.485661 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.485664 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.485681 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.485733 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.486305 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.486313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.486321 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.486407 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.486457 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6204,"thread_start_us":7600,"threads_started":1}
I20260811 06:23:37.487751 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.476841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.7.4.130:35531)
0811 06:23:37.484488 (+ 7647us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.484498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.484531 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:23:37.484614 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.484624 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.484835 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.485051 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.486013 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.486020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.486033 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.486060 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.486121 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.486142 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.487248 (+ 1106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.487339 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.487433 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7469,"thread_start_us":7354,"threads_started":1}
I20260811 06:23:37.488106 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.476327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33729 (local address 127.0.0.1:52498)
0811 06:23:37.483281 (+ 6954us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.483327 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.483418 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.485985 (+ 2567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.486007 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.486029 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.486817 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.486821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.486836 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.486879 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.487489 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.487494 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.487499 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.487613 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.487659 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6812,"thread_start_us":6654,"threads_started":1}
I20260811 06:23:37.487792 12502 tablet_service.cc:1467] Processing CreateTablet for tablet bca597ab814f46359823df51669a070d (DEFAULT_TABLE table=test-table [id=df6a46526cb946daa3eabb30bc06c0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:37.489233 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.477339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.7.4.131:33729)
0811 06:23:37.485114 (+ 7775us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.485122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.485155 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:23:37.485232 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.485240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.485814 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.485971 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.487201 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.487208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.487231 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.487255 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.487331 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.487353 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.488492 (+ 1139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.488585 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.488677 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7594,"thread_start_us":7410,"threads_started":1}
I20260811 06:23:37.489363 12502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca597ab814f46359823df51669a070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:37.491140 12644 tablet_service.cc:1467] Processing CreateTablet for tablet bca597ab814f46359823df51669a070d (DEFAULT_TABLE table=test-table [id=df6a46526cb946daa3eabb30bc06c0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:37.492530 12644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca597ab814f46359823df51669a070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:37.492692 12785 tablet_service.cc:1467] Processing CreateTablet for tablet bca597ab814f46359823df51669a070d (DEFAULT_TABLE table=test-table [id=df6a46526cb946daa3eabb30bc06c0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:37.493999 12785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bca597ab814f46359823df51669a070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:37.501123 12876 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Bootstrap starting.
I20260811 06:23:37.504854 12876 tablet_bootstrap.cc:654] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:37.505795 12876 log.cc:824] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:37.506735 12877 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Bootstrap starting.
I20260811 06:23:37.508299 12879 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Bootstrap starting.
I20260811 06:23:37.509908 12876 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: No bootstrap required, opened a new log
I20260811 06:23:37.510159 12876 ts_tablet_manager.cc:1397] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 06:23:37.510432 12877 tablet_bootstrap.cc:654] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:37.511696 12877 log.cc:824] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:37.512713 12879 tablet_bootstrap.cc:654] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:37.513936 12879 log.cc:824] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:37.514778 12877 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: No bootstrap required, opened a new log
I20260811 06:23:37.515036 12877 ts_tablet_manager.cc:1397] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:23:37.516947 12879 tablet_bootstrap.cc:492] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: No bootstrap required, opened a new log
I20260811 06:23:37.517198 12879 ts_tablet_manager.cc:1397] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 06:23:37.518759 12876 raft_consensus.cc:348] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.519106 12876 raft_consensus.cc:374] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:37.519238 12876 raft_consensus.cc:729] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02b3c4f03edd41be92e16681a1095412, State: Initialized, Role: FOLLOWER
I20260811 06:23:37.520068 12876 consensus_queue.cc:260] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.527603 12879 raft_consensus.cc:348] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.528041 12879 raft_consensus.cc:374] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:37.528158 12879 raft_consensus.cc:729] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9b14275a5d4f6f854911ef70ffce21, State: Initialized, Role: FOLLOWER
I20260811 06:23:37.528873 12879 consensus_queue.cc:260] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.530000 12571 heartbeater.cc:503] Master 127.7.4.190:44405 was elected leader, sending a full tablet report...
I20260811 06:23:37.530627 12876 ts_tablet_manager.cc:1428] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Time spent starting tablet: real 0.020s user 0.011s sys 0.010s
I20260811 06:23:37.531157 12877 raft_consensus.cc:348] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.531560 12877 raft_consensus.cc:374] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:37.531709 12877 raft_consensus.cc:729] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f65c058b8fe4596a64659e6a156bcac, State: Initialized, Role: FOLLOWER
I20260811 06:23:37.532366 12877 consensus_queue.cc:260] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.534320 12879 ts_tablet_manager.cc:1428] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260811 06:23:37.534710 12858 heartbeater.cc:503] Master 127.7.4.190:44405 was elected leader, sending a full tablet report...
I20260811 06:23:37.536885 12716 heartbeater.cc:503] Master 127.7.4.190:44405 was elected leader, sending a full tablet report...
I20260811 06:23:37.537524 12877 ts_tablet_manager.cc:1428] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Time spent starting tablet: real 0.022s user 0.019s sys 0.001s
W20260811 06:23:37.675923 12859 tablet.cc:2369] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:37.679399 12882 raft_consensus.cc:482] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:37.679682 12882 raft_consensus.cc:504] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.681377 12882 leader_election.cc:288] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f65c058b8fe4596a64659e6a156bcac (127.7.4.130:35531), 2f9b14275a5d4f6f854911ef70ffce21 (127.7.4.131:33729)
I20260811 06:23:37.684774 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.681768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:60353 (local address 127.7.4.130:35531)
0811 06:23:37.682146 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.682154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.682524 (+ 370us) server_negotiation.cc:407] Connection header received
0811 06:23:37.682740 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.682745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.682813 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.682914 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.684078 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.684084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.684089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.684110 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.684147 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.684159 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.684283 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.684375 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.684443 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260811 06:23:37.684872 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.681769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:57991 (local address 127.7.4.131:33729)
0811 06:23:37.681966 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.681972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.682926 (+ 954us) server_negotiation.cc:407] Connection header received
0811 06:23:37.683103 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.683108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.683166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.683275 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.684165 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.684170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.684175 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.684193 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.684231 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.684243 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.684371 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.684514 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.684597 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:23:37.685096 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.681814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33729 (local address 127.7.4.129:57991)
0811 06:23:37.682820 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.682834 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.682959 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.683323 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.683330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.683361 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.683970 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.683980 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.683993 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.684028 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.684337 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.684341 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.684347 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.684561 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.684629 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":577,"threads_started":1}
I20260811 06:23:37.684984 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.681579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:35531 (local address 127.7.4.129:60353)
0811 06:23:37.682405 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.682420 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:37.682582 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:37.682944 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:37.682949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:37.682976 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:37.683720 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:23:37.683724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:37.683745 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:23:37.683799 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:37.684261 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:37.684267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:37.684272 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:23:37.684425 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 06:23:37.684477 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":483,"threads_started":1}
I20260811 06:23:37.686108 12808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca597ab814f46359823df51669a070d" candidate_uuid: "02b3c4f03edd41be92e16681a1095412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9b14275a5d4f6f854911ef70ffce21" is_pre_election: true
I20260811 06:23:37.686160 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca597ab814f46359823df51669a070d" candidate_uuid: "02b3c4f03edd41be92e16681a1095412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f65c058b8fe4596a64659e6a156bcac" is_pre_election: true
I20260811 06:23:37.686587 12808 raft_consensus.cc:2454] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b3c4f03edd41be92e16681a1095412 in term 0.
I20260811 06:23:37.686684 12665 raft_consensus.cc:2454] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b3c4f03edd41be92e16681a1095412 in term 0.
I20260811 06:23:37.687383 12454 leader_election.cc:302] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b3c4f03edd41be92e16681a1095412, 2f9b14275a5d4f6f854911ef70ffce21; no voters:
I20260811 06:23:37.687762 12882 raft_consensus.cc:2781] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:37.687907 12882 raft_consensus.cc:482] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:37.688030 12882 raft_consensus.cc:3037] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:37.690392 12882 raft_consensus.cc:504] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.690940 12882 leader_election.cc:288] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [CANDIDATE]: Term 1 election: Requested vote from peers 6f65c058b8fe4596a64659e6a156bcac (127.7.4.130:35531), 2f9b14275a5d4f6f854911ef70ffce21 (127.7.4.131:33729)
I20260811 06:23:37.691433 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca597ab814f46359823df51669a070d" candidate_uuid: "02b3c4f03edd41be92e16681a1095412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f65c058b8fe4596a64659e6a156bcac"
I20260811 06:23:37.691454 12808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bca597ab814f46359823df51669a070d" candidate_uuid: "02b3c4f03edd41be92e16681a1095412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9b14275a5d4f6f854911ef70ffce21"
I20260811 06:23:37.691649 12665 raft_consensus.cc:3037] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:37.691649 12808 raft_consensus.cc:3037] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:37.694881 12665 raft_consensus.cc:2454] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b3c4f03edd41be92e16681a1095412 in term 1.
I20260811 06:23:37.694881 12808 raft_consensus.cc:2454] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b3c4f03edd41be92e16681a1095412 in term 1.
I20260811 06:23:37.695343 12455 leader_election.cc:302] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 02b3c4f03edd41be92e16681a1095412, 2f9b14275a5d4f6f854911ef70ffce21, 6f65c058b8fe4596a64659e6a156bcac; no voters:
I20260811 06:23:37.695634 12882 raft_consensus.cc:2781] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:37.696043 12882 raft_consensus.cc:686] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 1 LEADER]: Becoming Leader. State: Replica: 02b3c4f03edd41be92e16681a1095412, State: Running, Role: LEADER
I20260811 06:23:37.696477 12882 consensus_queue.cc:237] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } }
I20260811 06:23:37.700757 12370 catalog_manager.cc:5495] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02b3c4f03edd41be92e16681a1095412 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "02b3c4f03edd41be92e16681a1095412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02b3c4f03edd41be92e16681a1095412" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 } health_report { overall_health: UNKNOWN } } }
W20260811 06:23:37.716575 12717 tablet.cc:2369] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:37.722424 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.720408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43612 (local address 127.7.4.129:39483)
0811 06:23:37.720575 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.720579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.720799 (+ 220us) server_negotiation.cc:407] Connection header received
0811 06:23:37.720956 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.720959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.721032 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.721113 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.721864 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.721868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.721872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.721886 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.721915 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.721926 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.722012 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.722082 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.722125 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:23:37.730084 12665 raft_consensus.cc:1261] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 1 FOLLOWER]: Refusing update from remote peer 02b3c4f03edd41be92e16681a1095412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:37.730113 12808 raft_consensus.cc:1261] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 1 FOLLOWER]: Refusing update from remote peer 02b3c4f03edd41be92e16681a1095412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:37.730983 12887 consensus_queue.cc:1035] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f65c058b8fe4596a64659e6a156bcac" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 35531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:37.731343 12882 consensus_queue.cc:1035] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9b14275a5d4f6f854911ef70ffce21" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:37.742223 12891 mvcc.cc:205] Tried to move back new op lower bound from 7317214895013654528 to 7317214894888316928. Current Snapshot: MvccSnapshot[applied={T|T < 7317214895013654528}]
I20260811 06:23:37.742376 12890 mvcc.cc:205] Tried to move back new op lower bound from 7317214895013654528 to 7317214894888316928. Current Snapshot: MvccSnapshot[applied={T|T < 7317214895013654528}]
I20260811 06:23:37.747982 12892 mvcc.cc:205] Tried to move back new op lower bound from 7317214895013654528 to 7317214894888316928. Current Snapshot: MvccSnapshot[applied={T|T < 7317214895013654528 or (T in {7317214895013654528})}]
I20260811 06:23:37.755267 12370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:37.755564 12370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:37.758862 12370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f471cacc6f34f03bf7f12ae0dea3144: Sending DeleteTablet for 3 replicas of tablet bca597ab814f46359823df51669a070d
I20260811 06:23:37.759956 12502 tablet_service.cc:1514] Processing DeleteTablet for tablet bca597ab814f46359823df51669a070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:37 UTC) from {username='slave'} at 127.0.0.1:43600
I20260811 06:23:37.760345 12644 tablet_service.cc:1514] Processing DeleteTablet for tablet bca597ab814f46359823df51669a070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:37 UTC) from {username='slave'} at 127.0.0.1:45518
I20260811 06:23:37.760332 12785 tablet_service.cc:1514] Processing DeleteTablet for tablet bca597ab814f46359823df51669a070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:37 UTC) from {username='slave'} at 127.0.0.1:52498
I20260811 06:23:37.762498 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12435
I20260811 06:23:37.770299 12902 tablet_replica.cc:330] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: stopping tablet replica
I20260811 06:23:37.770812 12902 raft_consensus.cc:2229] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:37.772951 12902 raft_consensus.cc:2258] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:37.777004 12902 ts_tablet_manager.cc:1905] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:37.779716 12903 tablet_replica.cc:330] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: stopping tablet replica
I20260811 06:23:37.780229 12903 raft_consensus.cc:2229] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:37.780761 12903 raft_consensus.cc:2258] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:37.780983 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.766769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.7.4.129:39483)
0811 06:23:37.767313 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.767322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.778056 (+ 10734us) server_negotiation.cc:407] Connection header received
0811 06:23:37.778504 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.778510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.778570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.778640 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.779987 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.779993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.779998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.780017 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.780053 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.780069 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.780182 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.780561 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.780630 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260811 06:23:37.782085 12566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:37.783113 12903 ts_tablet_manager.cc:1905] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:37.786180 12902 ts_tablet_manager.cc:1918] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:37.786381 12902 log.cc:1196] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/bca597ab814f46359823df51669a070d
I20260811 06:23:37.786892 12902 ts_tablet_manager.cc:1939] T bca597ab814f46359823df51669a070d P 6f65c058b8fe4596a64659e6a156bcac: Deleting consensus metadata
I20260811 06:23:37.788475 12355 catalog_manager.cc:4841] TS 6f65c058b8fe4596a64659e6a156bcac (127.7.4.130:35531): tablet bca597ab814f46359823df51669a070d (table test-table [id=df6a46526cb946daa3eabb30bc06c0da]) successfully deleted
I20260811 06:23:37.791553 12903 ts_tablet_manager.cc:1918] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:37.791733 12903 log.cc:1196] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/bca597ab814f46359823df51669a070d
I20260811 06:23:37.792289 12903 ts_tablet_manager.cc:1939] T bca597ab814f46359823df51669a070d P 2f9b14275a5d4f6f854911ef70ffce21: Deleting consensus metadata
I20260811 06:23:37.793931 12354 catalog_manager.cc:4841] TS 2f9b14275a5d4f6f854911ef70ffce21 (127.7.4.131:33729): tablet bca597ab814f46359823df51669a070d (table test-table [id=df6a46526cb946daa3eabb30bc06c0da]) successfully deleted
I20260811 06:23:37.876905 12901 tablet_replica.cc:330] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: stopping tablet replica
I20260811 06:23:37.877379 12901 raft_consensus.cc:2229] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:37.878077 12901 raft_consensus.cc:2258] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:37.879952 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12435
I20260811 06:23:37.879941 12901 ts_tablet_manager.cc:1905] T bca597ab814f46359823df51669a070d P 02b3c4f03edd41be92e16681a1095412: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 06:23:37.900460 12355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.4.129:39483 (error 108)
I20260811 06:23:37.900539 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12576
I20260811 06:23:37.902611 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.902111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43628)
0811 06:23:37.902295 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.902446 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 06:23:37.902900 12355 catalog_manager.cc:4584] TS 02b3c4f03edd41be92e16681a1095412 (127.7.4.129:39483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bca597ab814f46359823df51669a070d: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
I20260811 06:23:37.903841 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.901499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.7.4.130:35531)
0811 06:23:37.901715 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:37.901721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:37.901741 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:23:37.901896 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:37.901903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:37.902035 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:37.902153 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:37.903198 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:37.903202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:37.903205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:37.903219 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:37.903249 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:37.903260 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:37.903405 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:37.903506 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 06:23:37.903553 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:23:37.904515 12711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:37.956243 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.955816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43642)
0811 06:23:37.955997 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.956076 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:23:37.957288 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:37.956943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43650)
0811 06:23:37.957088 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:37.957169 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:23:37.992133 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12576
I20260811 06:23:38.013602 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12720
I20260811 06:23:38.016585 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.014412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.7.4.131:33729)
0811 06:23:38.014601 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:38.014608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:38.014644 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:23:38.014817 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:38.014821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:38.014901 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:38.015009 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:38.015909 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:38.015915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:38.015920 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:38.015945 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:38.015977 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:38.015989 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:38.016125 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:38.016243 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 06:23:38.016306 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:23:38.017194 12852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:38.017428 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.016958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43658)
0811 06:23:38.017132 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:38.017219 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:23:38.018539 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.018152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43660)
0811 06:23:38.018307 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:38.018395 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:23:38.099074 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.098575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43674)
0811 06:23:38.098805 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:38.098910 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 06:23:38.100268 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.099886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39483 (local address 127.0.0.1:43678)
0811 06:23:38.100081 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:38.100150 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 06:23:38.105401 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12720
I20260811 06:23:38.126287 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12336
I20260811 06:23:38.127444 12403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:38.234534 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12336
2026-08-11T06:23:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2717 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 06:23:38.281370 7186 mini_oidc.cc:152] Starting JWKS server
I20260811 06:23:38.284720 7186 webserver.cc:460] Webserver started at https://127.0.0.1:41885/ using document root <none> and password file <none>
I20260811 06:23:38.285081 7186 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:23:38.285763 7186 webserver.cc:460] Webserver started at http://127.0.0.1:35031/ using document root <none> and password file <none>
2026-08-11T06:23:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:38Z Disabled control of system clock
I20260811 06:23:38.321101 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:43791
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:43791
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41885/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/testchainca.cert with env {}
W20260811 06:23:38.640764 12916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:38.641017 12916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:38.641184 12916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:38.648042 12916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:38.648162 12916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:38.648252 12916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:38.648351 12916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:38.648433 12916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:23:38.648521 12916 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:23:38.648660 12916 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41885/jwks.json
I20260811 06:23:38.658006 12916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:43791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:43791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41885/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:38.659209 12916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:38.660533 12916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:38.672359 12921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:38.672405 12922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:38.672622 12924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:38.673198 12916 server_base.cc:1034] running on GCE node
I20260811 06:23:38.673920 12916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:38.675171 12916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:38.676410 12916 hybrid_clock.cc:648] HybridClock initialized: now 1786429418676366 us; error 70 us; skew 500 ppm
I20260811 06:23:38.676759 12916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:38.678922 12916 webserver.cc:460] Webserver started at http://127.7.4.190:44849/ using document root <none> and password file <none>
I20260811 06:23:38.679615 12916 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:38.679762 12916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:38.680068 12916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:38.682862 12916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf"
format_stamp: "Formatted at 2026-08-11 06:23:38 on dist-test-slave-07c2"
server_key: "ab58fb5aa24a2a5122992d8c956931b1"
server_key_iv: "37baf6c2570d4b1cd8aa76cccad52214"
server_key_version: "encryptionkey@0"
I20260811 06:23:38.683709 12916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf"
format_stamp: "Formatted at 2026-08-11 06:23:38 on dist-test-slave-07c2"
server_key: "ab58fb5aa24a2a5122992d8c956931b1"
server_key_iv: "37baf6c2570d4b1cd8aa76cccad52214"
server_key_version: "encryptionkey@0"
I20260811 06:23:38.688723 12916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:23:38.692082 12930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:38.693262 12916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:38.693439 12916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf"
format_stamp: "Formatted at 2026-08-11 06:23:38 on dist-test-slave-07c2"
server_key: "ab58fb5aa24a2a5122992d8c956931b1"
server_key_iv: "37baf6c2570d4b1cd8aa76cccad52214"
server_key_version: "encryptionkey@0"
I20260811 06:23:38.693646 12916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:38.708988 12916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:38.709725 12916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:38.709998 12916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:38.728420 12916 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:43791
I20260811 06:23:38.728458 12985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:43791 every 8 connection(s)
I20260811 06:23:38.729707 12916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:38.732218 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 12916
I20260811 06:23:38.732427 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:38.733289 7186 external_mini_cluster.cc:1428] Setting key 8172d1708860007b08b307a6bf431b9b
I20260811 06:23:38.733793 12986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:38.747514 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.735345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.7.4.190:43791)
0811 06:23:38.736299 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:38.736309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:38.736351 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:23:38.738370 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:38.738390 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:38.738843 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:38.739183 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:38.739878 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.741287 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.743099 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.743838 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.743977 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:38.745337 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:38.745344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:38.745359 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:38.745868 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:38.745938 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:38.745961 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:38.746053 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:38.746283 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 06:23:38.746602 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":363,"threads_started":1}
I20260811 06:23:38.747512 12986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf: Bootstrap starting.
I20260811 06:23:38.750692 12986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:38.751199 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:23:38.751943 12986 log.cc:824] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:38.754679 12986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf: No bootstrap required, opened a new log
I20260811 06:23:38.761933 12986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } }
I20260811 06:23:38.762279 12986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:38.762418 12986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3cb94a4db8447f9e6c5ba4a235c3bf, State: Initialized, Role: FOLLOWER
I20260811 06:23:38.762940 12986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } }
I20260811 06:23:38.763199 12986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:38.763401 12986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:38.763582 12986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:38.765808 12986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } }
I20260811 06:23:38.766294 12986 leader_election.cc:302] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a3cb94a4db8447f9e6c5ba4a235c3bf; no voters:
I20260811 06:23:38.766865 12986 leader_election.cc:288] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:38.767020 12992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:38.767745 12992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [term 1 LEADER]: Becoming Leader. State: Replica: 1a3cb94a4db8447f9e6c5ba4a235c3bf, State: Running, Role: LEADER
I20260811 06:23:38.768265 12992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } }
I20260811 06:23:38.768953 12986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:38.771232 12993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } } }
I20260811 06:23:38.771615 12993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:38.771328 12994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a3cb94a4db8447f9e6c5ba4a235c3bf. Latest consensus state: current_term: 1 leader_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3cb94a4db8447f9e6c5ba4a235c3bf" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43791 } } }
I20260811 06:23:38.772934 12994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:38.777137 13000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:38.783659 13000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:38.792171 13000 catalog_manager.cc:1347] Generated new cluster ID: 2d05c768ea4643858b1170d51f09f140
I20260811 06:23:38.792274 13000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:38.823379 13000 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:38.824755 13000 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:38.842247 13000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a3cb94a4db8447f9e6c5ba4a235c3bf: Generated new TSK 0
I20260811 06:23:38.843058 13000 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:23:38.882045 13020 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 06:23:38.882264 13020 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:23:38.883446 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:38.861330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.7.4.190:43791)
0811 06:23:38.861522 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:38.861528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:38.862148 (+ 620us) server_negotiation.cc:407] Connection header received
0811 06:23:38.862464 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:38.862470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:38.862561 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:38.862708 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:23:38.863204 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.864356 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.865659 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.866427 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.866554 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:38.867119 (+ 565us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:23:38.882788 (+ 15669us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:23:38.883113 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:23:38.883569 12990 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 06:23:38.884874 13019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:38.861331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:43791 (local address 127.0.0.1:41016)
0811 06:23:38.862047 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:38.862066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:38.862203 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:38.862745 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:38.862748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:38.862758 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:23:38.863043 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:38.863050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.864484 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.864488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:38.865525 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:38.865536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:38.866570 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:38.866620 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:38.866805 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:38.866861 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:23:38.883032 (+ 16171us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:23:38.883330 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.4.190:43791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":404,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.4.190:43791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 06:23:38.886829 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 12916
I20260811 06:23:38.887971 12980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:38.981992 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 12916
2026-08-11T06:23:39Z chronyd exiting
I20260811 06:23:39.036468 7186 test_util.cc:175] -----------------------------------------------
I20260811 06:23:39.036618 7186 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429373903770-7186-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (759 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 06:23:39.040552 7186 mini_oidc.cc:152] Starting JWKS server
I20260811 06:23:39.042410 7186 webserver.cc:460] Webserver started at https://127.0.0.1:41765/ using document root <none> and password file <none>
I20260811 06:23:39.042668 7186 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:23:39.043460 7186 webserver.cc:460] Webserver started at http://127.0.0.1:35869/ using document root <none> and password file <none>
2026-08-11T06:23:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:39Z Disabled control of system clock
I20260811 06:23:39.075757 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37667
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37667
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41765/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/testchainca.cert with env {}
W20260811 06:23:39.394088 13027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:39.394433 13027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:39.394582 13027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:39.401662 13027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:39.401814 13027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:39.401894 13027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:39.402015 13027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:39.402096 13027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:23:39.402180 13027 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:23:39.402256 13027 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41765/jwks.json
I20260811 06:23:39.411726 13027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37667
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41765/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:39.412912 13027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:39.414182 13027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:39.425516 13032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:39.425477 13033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:39.425609 13035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:39.426384 13027 server_base.cc:1034] running on GCE node
I20260811 06:23:39.427136 13027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:39.428493 13027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:39.429745 13027 hybrid_clock.cc:648] HybridClock initialized: now 1786429419429696 us; error 37 us; skew 500 ppm
I20260811 06:23:39.430089 13027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:39.436267 13027 webserver.cc:460] Webserver started at http://127.7.4.190:40987/ using document root <none> and password file <none>
I20260811 06:23:39.436957 13027 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:39.437098 13027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:39.437448 13027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:39.440382 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "b6a194c98306465b8a0d16347ee8054c"
format_stamp: "Formatted at 2026-08-11 06:23:39 on dist-test-slave-07c2"
server_key: "ce7be64a051ff3b2382b54a5e481024c"
server_key_iv: "07f58dd3ceb5056f6c3f0dcf803614e5"
server_key_version: "encryptionkey@0"
I20260811 06:23:39.441164 13027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "b6a194c98306465b8a0d16347ee8054c"
format_stamp: "Formatted at 2026-08-11 06:23:39 on dist-test-slave-07c2"
server_key: "ce7be64a051ff3b2382b54a5e481024c"
server_key_iv: "07f58dd3ceb5056f6c3f0dcf803614e5"
server_key_version: "encryptionkey@0"
I20260811 06:23:39.446193 13027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:23:39.449462 13041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:39.450721 13027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:39.450906 13027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "b6a194c98306465b8a0d16347ee8054c"
format_stamp: "Formatted at 2026-08-11 06:23:39 on dist-test-slave-07c2"
server_key: "ce7be64a051ff3b2382b54a5e481024c"
server_key_iv: "07f58dd3ceb5056f6c3f0dcf803614e5"
server_key_version: "encryptionkey@0"
I20260811 06:23:39.451133 13027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:39.465687 13027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:39.466423 13027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:39.466718 13027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:39.484949 13027 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:37667
I20260811 06:23:39.484978 13098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:37667 every 8 connection(s)
I20260811 06:23:39.486234 13027 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:39.489677 13099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:39.493295 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13027
I20260811 06:23:39.493515 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:39.494144 7186 external_mini_cluster.cc:1428] Setting key e451cc602f35d99812017e8fceab2866
I20260811 06:23:39.500442 13099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c: Bootstrap starting.
I20260811 06:23:39.503813 13099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:39.504998 13099 log.cc:824] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:39.508424 13099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c: No bootstrap required, opened a new log
I20260811 06:23:39.508595 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:39.495981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.7.4.190:37667)
0811 06:23:39.496953 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:39.496963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:39.497005 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:23:39.498986 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:39.499005 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:39.499467 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:39.499761 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:39.500376 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.501698 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.503547 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.504280 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.504414 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:39.505753 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:39.505766 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:39.505788 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:39.506489 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:39.506564 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:39.506584 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:39.506676 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:39.506903 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 06:23:39.507201 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":391,"threads_started":1}
I20260811 06:23:39.512470 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:23:39.517866 13099 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } }
I20260811 06:23:39.518164 13099 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:39.518247 13099 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a194c98306465b8a0d16347ee8054c, State: Initialized, Role: FOLLOWER
I20260811 06:23:39.518685 13099 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } }
I20260811 06:23:39.518878 13099 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:39.519058 13099 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:39.519234 13099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:39.521492 13099 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } }
I20260811 06:23:39.521982 13099 leader_election.cc:302] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6a194c98306465b8a0d16347ee8054c; no voters:
I20260811 06:23:39.522540 13099 leader_election.cc:288] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:39.522691 13107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:39.524113 13107 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [term 1 LEADER]: Becoming Leader. State: Replica: b6a194c98306465b8a0d16347ee8054c, State: Running, Role: LEADER
I20260811 06:23:39.524621 13107 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } }
I20260811 06:23:39.524820 13099 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:39.527812 13108 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6a194c98306465b8a0d16347ee8054c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } } }
I20260811 06:23:39.528290 13108 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:39.528774 13109 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6a194c98306465b8a0d16347ee8054c. Latest consensus state: current_term: 1 leader_uuid: "b6a194c98306465b8a0d16347ee8054c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a194c98306465b8a0d16347ee8054c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37667 } } }
I20260811 06:23:39.529115 13109 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:39.530061 13117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:39.537679 13117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:39.546481 13117 catalog_manager.cc:1347] Generated new cluster ID: 00a2611ae46f4a848fdef4e09fb797ce
I20260811 06:23:39.546598 13117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:39.559964 13117 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:39.561861 13117 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:39.581071 13117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6a194c98306465b8a0d16347ee8054c: Generated new TSK 0
I20260811 06:23:39.582060 13117 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:23:39.621497 13133 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 06:23:39.621613 13133 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:23:39.622530 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:39.603260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.7.4.190:37667)
0811 06:23:39.603502 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:39.603508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:39.603894 (+ 386us) server_negotiation.cc:407] Connection header received
0811 06:23:39.604118 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:39.604121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:39.604185 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:39.604337 (+ 152us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:23:39.604861 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.605589 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.607679 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.608117 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.608222 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:39.608798 (+ 576us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:23:39.622072 (+ 13274us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:23:39.622331 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 06:23:39.622674 13103 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 06:23:39.622891 13132 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:39.603095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37667 (local address 127.0.0.1:55316)
0811 06:23:39.603787 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:39.603806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:39.603962 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:39.604391 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:39.604399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:39.604414 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:23:39.604711 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:39.604723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.605742 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.605748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:39.607542 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:39.607558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:39.608242 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:39.608311 (+ 69us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:39.608510 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:39.608569 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:23:39.622256 (+ 13687us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:23:39.622500 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.4.190:37667: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":354,"threads_started":1}
I20260811 06:23:39.624241 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13027
I20260811 06:23:39.625407 13094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:39.719918 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13027
2026-08-11T06:23:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (740 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 06:23:39.780589 7186 mini_oidc.cc:152] Starting JWKS server
I20260811 06:23:39.782553 7186 webserver.cc:460] Webserver started at https://127.0.0.1:43293/ using document root <none> and password file <none>
I20260811 06:23:39.782752 7186 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:23:39.783404 7186 webserver.cc:460] Webserver started at http://127.0.0.1:44019/ using document root <none> and password file <none>
2026-08-11T06:23:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:39Z Disabled control of system clock
I20260811 06:23:39.818827 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:39831
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:39831
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43293/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:23:40.134548 13141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:40.134855 13141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:40.134974 13141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:40.142263 13141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:40.142373 13141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:40.142411 13141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:40.142491 13141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:40.142578 13141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:23:40.142688 13141 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:23:40.142755 13141 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43293/jwks.json
I20260811 06:23:40.151914 13141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:39831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43293/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:40.153362 13141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:40.155129 13141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:40.166344 13147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:40.166467 13146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:40.166455 13149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:40.167400 13141 server_base.cc:1034] running on GCE node
I20260811 06:23:40.168123 13141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:40.169489 13141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:40.170691 13141 hybrid_clock.cc:648] HybridClock initialized: now 1786429420170630 us; error 43 us; skew 500 ppm
I20260811 06:23:40.171029 13141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:40.177310 13141 webserver.cc:460] Webserver started at http://127.7.4.190:45611/ using document root <none> and password file <none>
I20260811 06:23:40.177989 13141 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:40.178112 13141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:40.178455 13141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:40.181455 13141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "02a5745d172e4499816ad4c8792a942f"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "574eb90ea73f85ad28000d5beabc4760"
server_key_iv: "60ba7a89e3356e0736f920733aaa28e6"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.182250 13141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "02a5745d172e4499816ad4c8792a942f"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "574eb90ea73f85ad28000d5beabc4760"
server_key_iv: "60ba7a89e3356e0736f920733aaa28e6"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.187224 13141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:23:40.190550 13155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:40.191795 13141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:23:40.191977 13141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "02a5745d172e4499816ad4c8792a942f"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "574eb90ea73f85ad28000d5beabc4760"
server_key_iv: "60ba7a89e3356e0736f920733aaa28e6"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.192183 13141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:40.205612 13141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:40.206369 13141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:40.206661 13141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:40.225169 13141 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:39831
I20260811 06:23:40.225188 13209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:39831 every 8 connection(s)
I20260811 06:23:40.226516 13141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:40.229997 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13141
I20260811 06:23:40.230211 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:40.230401 13211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:40.231093 7186 external_mini_cluster.cc:1428] Setting key 7d6493248d15af87022a2771c0966d4a
I20260811 06:23:40.243739 13211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f: Bootstrap starting.
I20260811 06:23:40.245258 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:40.233021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.7.4.190:39831)
0811 06:23:40.234128 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:40.234138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:40.234178 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:23:40.236495 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:40.236515 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:40.236955 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:40.237271 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:40.237819 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.239025 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.240927 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.241589 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.241713 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:40.242920 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:40.242927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:40.242938 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:40.243524 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:40.243594 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:40.243614 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:40.243703 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:40.243964 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 06:23:40.244267 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":458,"threads_started":1}
I20260811 06:23:40.246912 13211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:40.248127 13211 log.cc:824] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:40.249011 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:23:40.251170 13211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f: No bootstrap required, opened a new log
I20260811 06:23:40.258628 13211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } }
I20260811 06:23:40.258934 13211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:40.259004 13211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a5745d172e4499816ad4c8792a942f, State: Initialized, Role: FOLLOWER
I20260811 06:23:40.259547 13211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } }
I20260811 06:23:40.259750 13211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:40.259843 13211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:40.259953 13211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:40.262125 13211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } }
I20260811 06:23:40.262523 13211 leader_election.cc:302] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02a5745d172e4499816ad4c8792a942f; no voters:
I20260811 06:23:40.263034 13211 leader_election.cc:288] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:40.263203 13216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:40.263974 13216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [term 1 LEADER]: Becoming Leader. State: Replica: 02a5745d172e4499816ad4c8792a942f, State: Running, Role: LEADER
I20260811 06:23:40.264463 13216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } }
I20260811 06:23:40.265100 13211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:40.267613 13218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02a5745d172e4499816ad4c8792a942f. Latest consensus state: current_term: 1 leader_uuid: "02a5745d172e4499816ad4c8792a942f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } } }
I20260811 06:23:40.267968 13218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:40.268756 13217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02a5745d172e4499816ad4c8792a942f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a5745d172e4499816ad4c8792a942f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 39831 } } }
I20260811 06:23:40.269145 13217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:40.272157 13222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:40.277066 13222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:40.285616 13222 catalog_manager.cc:1347] Generated new cluster ID: 3409cc31d8ad44aa8cd3d32ba418b7ff
I20260811 06:23:40.285725 13222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:40.310474 13222 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:40.312258 13222 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:40.332026 13222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02a5745d172e4499816ad4c8792a942f: Generated new TSK 0
I20260811 06:23:40.333012 13222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:23:40.355715 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:40.348085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.7.4.190:39831)
0811 06:23:40.348269 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:40.348275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:40.348294 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:23:40.348587 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:40.348592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:40.348658 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:40.348807 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:23:40.349294 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.350224 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.351809 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.352257 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.352394 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:40.352922 (+ 528us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:23:40.354771 (+ 1849us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43293/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:23:40.355172 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43293/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 06:23:40.355741 13241 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:40.347612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:39831 (local address 127.0.0.1:43888)
0811 06:23:40.348191 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:40.348210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:40.348358 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:40.348825 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:40.348830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:40.348842 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:23:40.349126 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:40.349136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.350394 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.350404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:40.351656 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:40.351670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.352385 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.352480 (+ 95us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:40.352634 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:40.352666 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:23:40.355044 (+ 2378us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43293/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:23:40.355315 (+ 271us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.4.190:39831: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43293/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":308,"threads_started":1}
W20260811 06:23:40.355863 13214 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43293/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260811 06:23:40.357213 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13141
I20260811 06:23:40.358713 13205 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 06:23:40.367954 13242 webserver.cc:533] Webserver: sslize failed:
E20260811 06:23:40.368049 13242 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:23:40.449450 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13141
2026-08-11T06:23:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (727 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T06:23:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:40Z Disabled control of system clock
I20260811 06:23:40.542702 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36509
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:36509
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/testchainca.cert with env {}
W20260811 06:23:40.863606 13247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:40.863883 13247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:40.863976 13247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:40.870975 13247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:40.871085 13247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:40.871153 13247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:40.871266 13247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:40.871385 13247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:23:40.871491 13247 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:23:40.871562 13247 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 06:23:40.881495 13247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:36509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:40.882771 13247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:40.884033 13247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:40.895709 13256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:40.895766 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:40.895794 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:40.896430 13247 server_base.cc:1034] running on GCE node
I20260811 06:23:40.897157 13247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:40.898486 13247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:40.899711 13247 hybrid_clock.cc:648] HybridClock initialized: now 1786429420899671 us; error 63 us; skew 500 ppm
I20260811 06:23:40.900055 13247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:23:40.902325 13247 webserver.cc:460] Webserver started at http://127.7.4.190:44675/ using document root <none> and password file <none>
I20260811 06:23:40.903014 13247 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:40.903148 13247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:40.903534 13247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:40.906363 13247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "26b099d49a85457e96a986e312b885e2"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "491723357662ec74a639fe9abd23daee"
server_key_iv: "fe84cac8cd1d8c04b5890cc8b520991f"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.907171 13247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "26b099d49a85457e96a986e312b885e2"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "491723357662ec74a639fe9abd23daee"
server_key_iv: "fe84cac8cd1d8c04b5890cc8b520991f"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.912766 13247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:23:40.916002 13265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:40.917284 13247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:40.917471 13247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "26b099d49a85457e96a986e312b885e2"
format_stamp: "Formatted at 2026-08-11 06:23:40 on dist-test-slave-07c2"
server_key: "491723357662ec74a639fe9abd23daee"
server_key_iv: "fe84cac8cd1d8c04b5890cc8b520991f"
server_key_version: "encryptionkey@0"
I20260811 06:23:40.917678 13247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:40.928520 13247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:40.929286 13247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:40.929570 13247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:40.947172 13247 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:36509
I20260811 06:23:40.947206 13319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:36509 every 8 connection(s)
I20260811 06:23:40.948421 13247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:40.952010 13320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:40.952880 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13247
I20260811 06:23:40.953099 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:40.953796 7186 external_mini_cluster.cc:1428] Setting key 633d091f5c48c65e8c13d4b09709f0c4
I20260811 06:23:40.965022 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2: Bootstrap starting.
I20260811 06:23:40.967339 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:40.955708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.7.4.190:36509)
0811 06:23:40.956874 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:40.956888 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:40.956931 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:23:40.958562 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:40.958585 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:40.959077 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:40.959539 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:40.960132 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.961378 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.962721 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:40.963508 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:40.963704 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:40.964940 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:40.964947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:40.964962 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:23:40.965558 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:40.965636 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:23:40.965658 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:40.965792 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:40.966017 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 06:23:40.966328 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":485,"threads_started":1}
I20260811 06:23:40.968809 13320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:40.970019 13320 log.cc:824] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:40.971566 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:23:40.973507 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2: No bootstrap required, opened a new log
I20260811 06:23:40.981642 13320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } }
I20260811 06:23:40.982065 13320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:40.982151 13320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26b099d49a85457e96a986e312b885e2, State: Initialized, Role: FOLLOWER
I20260811 06:23:40.982637 13320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } }
I20260811 06:23:40.982831 13320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:40.982930 13320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:40.983062 13320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:40.985275 13320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } }
I20260811 06:23:40.985733 13320 leader_election.cc:302] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26b099d49a85457e96a986e312b885e2; no voters:
I20260811 06:23:40.986296 13320 leader_election.cc:288] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:40.986479 13326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:40.987226 13326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [term 1 LEADER]: Becoming Leader. State: Replica: 26b099d49a85457e96a986e312b885e2, State: Running, Role: LEADER
I20260811 06:23:40.987792 13326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } }
I20260811 06:23:40.988481 13320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:40.990674 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26b099d49a85457e96a986e312b885e2. Latest consensus state: current_term: 1 leader_uuid: "26b099d49a85457e96a986e312b885e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } } }
I20260811 06:23:40.990710 13327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26b099d49a85457e96a986e312b885e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26b099d49a85457e96a986e312b885e2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36509 } } }
I20260811 06:23:40.991127 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:40.991178 13327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:40.992295 13335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:40.997561 13335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:41.006017 13335 catalog_manager.cc:1347] Generated new cluster ID: be2021337d6d48638f735e368858ccf7
I20260811 06:23:41.006177 13335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:41.046207 13335 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:41.048153 13335 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:41.055527 13335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26b099d49a85457e96a986e312b885e2: Generated new TSK 0
I20260811 06:23:41.056514 13335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:23:41.128801 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:41.077030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.7.4.190:36509)
0811 06:23:41.077263 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:41.077272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:41.077757 (+ 485us) server_negotiation.cc:407] Connection header received
0811 06:23:41.077982 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:41.077986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:41.078050 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:41.078185 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:23:41.078754 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:41.079547 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:41.081586 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:41.082026 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:41.082122 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:41.082677 (+ 555us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:23:41.128241 (+ 45564us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:23:41.128565 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46204: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:23:41.128963 13324 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46204: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 06:23:41.129603 13353 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:41.077019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:36509 (local address 127.0.0.1:46204)
0811 06:23:41.077649 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:41.077668 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:41.077821 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:41.078243 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:41.078248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:41.078261 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:23:41.078599 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:41.078611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:41.079684 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:41.079691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:41.081418 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:41.081434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:41.082143 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:41.082218 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:41.082408 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:41.082452 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:23:41.128547 (+ 46095us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:23:41.128863 (+ 316us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.4.190:36509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":330,"threads_started":1}
I20260811 06:23:41.131279 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13247
I20260811 06:23:41.132503 13315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:41.231590 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13247
2026-08-11T06:23:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (762 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (417 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (418 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:42 dist-test-slave-07c2 krb5kdc[13369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:42 dist-test-slave-07c2 krb5kdc[13369](info): set up 2 sockets
Aug 11 06:23:42 dist-test-slave-07c2 krb5kdc[13369](info): commencing operation
krb5kdc: starting...
W20260811 06:23:43.214500 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:43 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:23:43.478410 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:40091
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:43.817180 13382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:43.817464 13382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:43.817615 13382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:43.824703 13382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:43.824795 13382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:43.824844 13382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:43.824924 13382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:43.825006 13382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:43.834493 13382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:40091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:43.835803 13382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:43.837082 13382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:43.848697 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:43.848742 13390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:43.849181 13393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:43.849663 13382 server_base.cc:1034] running on GCE node
I20260811 06:23:43.850428 13382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:43.851759 13382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:43.852994 13382 hybrid_clock.cc:648] HybridClock initialized: now 1786429423852955 us; error 41 us; skew 500 ppm
I20260811 06:23:43.853410 13382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:43 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:43.856482 13382 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:23:43.857880 13382 webserver.cc:460] Webserver started at http://127.7.4.190:40565/ using document root <none> and password file <none>
I20260811 06:23:43.858531 13382 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:43.858678 13382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:43.858994 13382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:43.861889 13382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "baa0a7f2562b49bb97001a7a089773b2"
format_stamp: "Formatted at 2026-08-11 06:23:43 on dist-test-slave-07c2"
server_key: "034962d174df9a4082bfb8e8a8410e64"
server_key_iv: "6c5207f6d5a841e2e4cae0fd1da1b704"
server_key_version: "encryptionkey@0"
I20260811 06:23:43.862650 13382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "baa0a7f2562b49bb97001a7a089773b2"
format_stamp: "Formatted at 2026-08-11 06:23:43 on dist-test-slave-07c2"
server_key: "034962d174df9a4082bfb8e8a8410e64"
server_key_iv: "6c5207f6d5a841e2e4cae0fd1da1b704"
server_key_version: "encryptionkey@0"
I20260811 06:23:43.867622 13382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:23:43.870816 13400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:43.872076 13382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:43.872260 13382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "baa0a7f2562b49bb97001a7a089773b2"
format_stamp: "Formatted at 2026-08-11 06:23:43 on dist-test-slave-07c2"
server_key: "034962d174df9a4082bfb8e8a8410e64"
server_key_iv: "6c5207f6d5a841e2e4cae0fd1da1b704"
server_key_version: "encryptionkey@0"
I20260811 06:23:43.872443 13382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:43.906037 13382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:43.909857 13382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:43.910151 13382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:43.928318 13382 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:40091
I20260811 06:23:43.928359 13454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:40091 every 8 connection(s)
I20260811 06:23:43.929563 13382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:43.933003 13455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:43.934937 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13382
I20260811 06:23:43.935175 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:43.935981 7186 external_mini_cluster.cc:1428] Setting key 296348fb5ef5b06aa89592c2826b244e
I20260811 06:23:43.944372 13455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2: Bootstrap starting.
I20260811 06:23:43.947137 13455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:43.948108 13455 log.cc:824] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:23:43 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:43.951056 13455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2: No bootstrap required, opened a new log
I20260811 06:23:43.957592 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:43.938073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.7.4.190:40091)
0811 06:23:43.938817 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:43.938828 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:43.938868 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:23:43.940611 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:43.940631 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:43.940966 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:43.941313 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:43.942735 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:43.944048 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:43.945345 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:43.946008 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:43.946206 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:43.950342 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:43.950352 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:43.950369 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:43.950417 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:43.953717 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:43.954488 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:43.954493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:43.954498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:43.954666 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:43.955096 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:43.955099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:43.955101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:43.955659 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:43.955888 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:43.956318 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 06:23:43.956590 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":322,"threads_started":1}
I20260811 06:23:43.959966 13455 raft_consensus.cc:348] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } }
I20260811 06:23:43.960391 13455 raft_consensus.cc:374] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:43.960515 13455 raft_consensus.cc:729] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0a7f2562b49bb97001a7a089773b2, State: Initialized, Role: FOLLOWER
I20260811 06:23:43.961035 13455 consensus_queue.cc:260] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } }
I20260811 06:23:43.961311 13455 raft_consensus.cc:388] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:43.961460 13455 raft_consensus.cc:482] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:43.961627 13455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:43.963780 13455 raft_consensus.cc:504] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } }
I20260811 06:23:43.964234 13455 leader_election.cc:302] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baa0a7f2562b49bb97001a7a089773b2; no voters:
I20260811 06:23:43.964802 13455 leader_election.cc:288] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:43.964922 13460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:43.966086 13460 raft_consensus.cc:686] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [term 1 LEADER]: Becoming Leader. State: Replica: baa0a7f2562b49bb97001a7a089773b2, State: Running, Role: LEADER
I20260811 06:23:43.966532 13460 consensus_queue.cc:237] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } }
I20260811 06:23:43.966773 13455 sys_catalog.cc:564] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:43.969637 13461 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baa0a7f2562b49bb97001a7a089773b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } } }
I20260811 06:23:43.969587 13462 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baa0a7f2562b49bb97001a7a089773b2. Latest consensus state: current_term: 1 leader_uuid: "baa0a7f2562b49bb97001a7a089773b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa0a7f2562b49bb97001a7a089773b2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 40091 } } }
I20260811 06:23:43.970032 13461 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:43.970067 13462 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:43.972242 13469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:43.976902 13469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:43.985984 13469 catalog_manager.cc:1347] Generated new cluster ID: 70996f6f181642e6beb9789dbdbb2e75
I20260811 06:23:43.986089 13469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:44.013100 13469 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:44.014428 13469 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:44.025204 13469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baa0a7f2562b49bb97001a7a089773b2: Generated new TSK 0
I20260811 06:23:44.025947 13469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:23:44.096588 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:44.426607 13482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:44.426895 13482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:44.426988 13482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:44.433990 13482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:44.434104 13482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:44.434226 13482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:44.443538 13482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:44.444739 13482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:44.445947 13482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:44.458573 13491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:44.458657 13489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:44.458680 13488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:44.459844 13482 server_base.cc:1034] running on GCE node
I20260811 06:23:44.460475 13482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:44.461345 13482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:44.462569 13482 hybrid_clock.cc:648] HybridClock initialized: now 1786429424462535 us; error 46 us; skew 500 ppm
I20260811 06:23:44.462872 13482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:44 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:44.470471 13482 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:23:44.472100 13482 webserver.cc:460] Webserver started at http://127.7.4.129:40649/ using document root <none> and password file <none>
I20260811 06:23:44.472811 13482 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:44.472954 13482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:44.473263 13482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:44.476184 13482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "0ca406506c994fffa1b8d76ed627b44f"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "26721605fa133bcc779bbf8d660b044a"
server_key_iv: "e6d49ea4939ba40103dcc8077769d306"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.477157 13482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "0ca406506c994fffa1b8d76ed627b44f"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "26721605fa133bcc779bbf8d660b044a"
server_key_iv: "e6d49ea4939ba40103dcc8077769d306"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.481979 13482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:23:44.485284 13499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:44.486485 13482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:23:44.486656 13482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "0ca406506c994fffa1b8d76ed627b44f"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "26721605fa133bcc779bbf8d660b044a"
server_key_iv: "e6d49ea4939ba40103dcc8077769d306"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.486858 13482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:44.497763 13482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:44.501395 13482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:44.501695 13482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:44.502669 13482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:44.504010 13482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:44.504124 13482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:44.504297 13482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:44.504370 13482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:44.537405 13482 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:42837
I20260811 06:23:44.537446 13620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:42837 every 8 connection(s)
I20260811 06:23:44.538728 13482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:44.543248 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13482
I20260811 06:23:44.543524 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:44.544370 7186 external_mini_cluster.cc:1428] Setting key 0c583c2fd03911e65db195a74c212e60
Aug 11 06:23:44 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:44.562496 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:44.541473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:51163 (local address 127.7.4.190:40091)
0811 06:23:44.541932 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:44.541937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:44.542968 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 06:23:44.545498 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:44.545506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:44.545606 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:44.545701 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:44.547758 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:44.548481 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:44.550971 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:44.551461 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:44.551566 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:44.555872 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:44.555881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:44.555888 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:44.555935 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:44.558957 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:44.559928 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:44.559933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:44.559935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:44.560052 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:44.560529 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:44.560532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:44.560534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:44.560787 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:44.560906 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:44.561877 (+ 971us) server_negotiation.cc:299] Negotiation successful
0811 06:23:44.562042 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":244,"threads_started":1}
I20260811 06:23:44.563915 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:44.542010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:40091 (local address 127.7.4.129:51163)
0811 06:23:44.542762 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:44.542820 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:44.545242 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:44.545991 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:44.546003 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:44.546690 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:44.547549 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:44.547581 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:44.548639 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:44.548646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:44.550801 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:44.550819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:44.551694 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:44.551702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:44.552037 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:44.552807 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:44.552842 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:44.555655 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:44.559093 (+ 3438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:44.559105 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:44.559124 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:44.559808 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:44.560158 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:44.560165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:44.560172 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:44.560418 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:44.560916 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:44.560927 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:44.561398 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 06:23:44.561776 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 06:23:44.562354 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":291,"threads_started":1}
I20260811 06:23:44.565546 13621 heartbeater.cc:347] Connected to a master server at 127.7.4.190:40091
I20260811 06:23:44.565871 13621 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:44.566661 13621 heartbeater.cc:511] Master 127.7.4.190:40091 requested a full tablet report, sending...
I20260811 06:23:44.568918 13419 ts_manager.cc:194] Registered new tserver with Master: 0ca406506c994fffa1b8d76ed627b44f (127.7.4.129:42837)
I20260811 06:23:44.570324 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:51163
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:23:44.606953 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:44.942119 13628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:44.942360 13628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:44.942447 13628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:44.949280 13628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:44.949375 13628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:44.949492 13628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:44.958423 13628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:44.959476 13628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:44.960809 13628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:44.973742 13634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:44.973830 13633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:44.973803 13636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:44.974712 13628 server_base.cc:1034] running on GCE node
I20260811 06:23:44.975204 13628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:44.976070 13628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:44.977303 13628 hybrid_clock.cc:648] HybridClock initialized: now 1786429424977247 us; error 44 us; skew 500 ppm
I20260811 06:23:44.977571 13628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:44 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:44.980661 13628 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:23:44.982174 13628 webserver.cc:460] Webserver started at http://127.7.4.130:45137/ using document root <none> and password file <none>
I20260811 06:23:44.982852 13628 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:44.982959 13628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:44.983206 13628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:44.986071 13628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "88d5be4b7b624edb9762c81e9ac01aa6"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "0daaaab60eace6820fb7bfab0cc334ae"
server_key_iv: "657bb47a9e11e7b8d48e4e28804d4a83"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.986809 13628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "88d5be4b7b624edb9762c81e9ac01aa6"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "0daaaab60eace6820fb7bfab0cc334ae"
server_key_iv: "657bb47a9e11e7b8d48e4e28804d4a83"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.991894 13628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:23:44.995057 13644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:44.996325 13628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:44.996482 13628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "88d5be4b7b624edb9762c81e9ac01aa6"
format_stamp: "Formatted at 2026-08-11 06:23:44 on dist-test-slave-07c2"
server_key: "0daaaab60eace6820fb7bfab0cc334ae"
server_key_iv: "657bb47a9e11e7b8d48e4e28804d4a83"
server_key_version: "encryptionkey@0"
I20260811 06:23:44.996706 13628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:45.025367 13628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:45.029405 13628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:45.029721 13628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:45.030675 13628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:45.032138 13628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:45.032243 13628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:45.032390 13628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:45.032480 13628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:45.065690 13628 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:33211
I20260811 06:23:45.065775 13765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:33211 every 8 connection(s)
I20260811 06:23:45.066989 13628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:45.072567 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13628
I20260811 06:23:45.072757 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:45.073320 7186 external_mini_cluster.cc:1428] Setting key 2780809c2486cca8259d958126e91e84
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:45.089265 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.069592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:56155 (local address 127.7.4.190:40091)
0811 06:23:45.070020 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.070024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.071058 (+ 1034us) server_negotiation.cc:407] Connection header received
0811 06:23:45.073347 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.073351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.073449 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.073554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.075253 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.076019 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.078575 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.078979 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.079071 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.083611 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.083615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.083618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.083649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.086193 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.086864 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.086870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.086873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.087020 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.087545 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.087550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.087555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.087814 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.087971 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.088654 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.088882 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":228,"threads_started":1}
I20260811 06:23:45.090435 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.070145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:40091 (local address 127.7.4.130:56155)
0811 06:23:45.070891 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:45.070933 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:45.073067 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:45.073822 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:45.073832 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:45.074362 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:45.075102 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:45.075122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.076199 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.076206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:45.078434 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:45.078452 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.079214 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.079223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:45.079561 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.080303 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:45.080336 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:45.083393 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:45.086341 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:45.086349 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:45.086362 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:45.086752 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:45.087133 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:45.087143 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:45.087147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:45.087431 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:45.087945 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:45.087953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:45.088270 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 06:23:45.088514 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 06:23:45.089030 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":288,"threads_started":1}
I20260811 06:23:45.091847 13766 heartbeater.cc:347] Connected to a master server at 127.7.4.190:40091
I20260811 06:23:45.092161 13766 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:45.093047 13766 heartbeater.cc:511] Master 127.7.4.190:40091 requested a full tablet report, sending...
I20260811 06:23:45.094722 13419 ts_manager.cc:194] Registered new tserver with Master: 88d5be4b7b624edb9762c81e9ac01aa6 (127.7.4.130:33211)
I20260811 06:23:45.095990 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:56155
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:23:45.130504 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:45.456583 13774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:45.456823 13774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:45.457000 13774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:45.463912 13774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:45.464030 13774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:45.464212 13774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:45.473352 13774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:45.474469 13774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:45.475821 13774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:45.488529 13780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:45.488576 13785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:45.488628 13781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:45.489380 13774 server_base.cc:1034] running on GCE node
I20260811 06:23:45.489952 13774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:45.490869 13774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:45.492092 13774 hybrid_clock.cc:648] HybridClock initialized: now 1786429425492054 us; error 31 us; skew 500 ppm
I20260811 06:23:45.492535 13774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.495777 13774 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:23:45.497119 13774 webserver.cc:460] Webserver started at http://127.7.4.131:35313/ using document root <none> and password file <none>
I20260811 06:23:45.497722 13774 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:45.497822 13774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:45.498082 13774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:45.501037 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "bfb34a7ab64d4ceeb25bd612113a49fd"
format_stamp: "Formatted at 2026-08-11 06:23:45 on dist-test-slave-07c2"
server_key: "7526cc3cf38ee77c68aef554100a9ed4"
server_key_iv: "5b024c725b1129ee0f883524e86f4eab"
server_key_version: "encryptionkey@0"
I20260811 06:23:45.501744 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "bfb34a7ab64d4ceeb25bd612113a49fd"
format_stamp: "Formatted at 2026-08-11 06:23:45 on dist-test-slave-07c2"
server_key: "7526cc3cf38ee77c68aef554100a9ed4"
server_key_iv: "5b024c725b1129ee0f883524e86f4eab"
server_key_version: "encryptionkey@0"
I20260811 06:23:45.506646 13774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:23:45.510071 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:45.511189 13774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:23:45.511379 13774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "bfb34a7ab64d4ceeb25bd612113a49fd"
format_stamp: "Formatted at 2026-08-11 06:23:45 on dist-test-slave-07c2"
server_key: "7526cc3cf38ee77c68aef554100a9ed4"
server_key_iv: "5b024c725b1129ee0f883524e86f4eab"
server_key_version: "encryptionkey@0"
I20260811 06:23:45.511581 13774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:45.541754 13774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:45.546132 13774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:45.546442 13774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:45.547484 13774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:45.548805 13774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:45.548913 13774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:45.549074 13774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:45.549158 13774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:45.573423 13621 heartbeater.cc:503] Master 127.7.4.190:40091 was elected leader, sending a full tablet report...
I20260811 06:23:45.586524 13774 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:41763
I20260811 06:23:45.586571 13913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:41763 every 8 connection(s)
I20260811 06:23:45.588217 13774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:45.596537 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13774
I20260811 06:23:45.596747 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:45.597162 7186 external_mini_cluster.cc:1428] Setting key 5f0ce616d9a4cd564284df7e3a20b4fe
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:23:45.608506 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.590857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:33237 (local address 127.7.4.190:40091)
0811 06:23:45.591396 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.591400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.592308 (+ 908us) server_negotiation.cc:407] Connection header received
0811 06:23:45.594067 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.594072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.594158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.594276 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.595717 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.596481 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.598293 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.598765 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.598901 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.602516 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.602521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.602525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.602559 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.605118 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.605922 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.605927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.605937 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.606055 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.606483 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.606488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.606492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.606766 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.606939 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.607867 (+ 928us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.608118 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":249,"threads_started":1}
I20260811 06:23:45.609444 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.591489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:40091 (local address 127.7.4.131:33237)
0811 06:23:45.592162 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:45.592206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:45.593807 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:45.594503 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:45.594511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:45.594985 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:45.595558 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:45.595575 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.596637 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.596641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:45.598128 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:45.598140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.599138 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.599143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:45.599402 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.600123 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:45.600153 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:45.602298 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:45.605269 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:45.605277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:45.605290 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:45.605797 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:45.606167 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:45.606172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:45.606175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:45.606366 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:45.606894 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:45.606903 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:45.607453 (+ 550us) client_negotiation.cc:769] Sending connection context
0811 06:23:45.607752 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:23:45.608298 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":284,"threads_started":1}
I20260811 06:23:45.610854 13914 heartbeater.cc:347] Connected to a master server at 127.7.4.190:40091
I20260811 06:23:45.611209 13914 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:45.612010 13914 heartbeater.cc:511] Master 127.7.4.190:40091 requested a full tablet report, sending...
I20260811 06:23:45.613466 13419 ts_manager.cc:194] Registered new tserver with Master: bfb34a7ab64d4ceeb25bd612113a49fd (127.7.4.131:41763)
I20260811 06:23:45.614288 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:33237
I20260811 06:23:45.625361 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/test-admin.keytab.
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.659034 7186 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 06:23:45.664633 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:45.662008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.7.4.129:42837)
0811 06:23:45.662688 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.662696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.662726 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:23:45.662790 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.662795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.663014 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.663160 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.664223 (+ 1063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34708: BlockingRecv error: recv got EOF from 127.0.0.1:34708 (error 108)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":379,"threads_started":1}
W20260811 06:23:45.665494 7186 init.cc:244] Error while opening credential cache '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc)
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.667522 7186 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.669076 7186 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:45.685168 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.670863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.7.4.129:42837)
0811 06:23:45.671085 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.671092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.671358 (+ 266us) server_negotiation.cc:407] Connection header received
0811 06:23:45.671587 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.671591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.671655 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.671765 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.672879 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.673885 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.675173 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.675891 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.676039 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.679797 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.679810 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.679825 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.679868 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.682520 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.683218 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.683225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.683227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.683420 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.683744 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.683747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.683749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.684109 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.684323 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.684600 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.684883 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 06:23:45.691138 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:45.689125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.7.4.129:42837)
0811 06:23:45.689288 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.689292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.689669 (+ 377us) server_negotiation.cc:407] Connection header received
0811 06:23:45.689798 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.689801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.689870 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.689948 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.690848 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34728: BlockingRecv error: recv got EOF from 127.0.0.1:34728 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 06:23:45.691395 7186 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.692982 7186 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.694264 7186 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:45.708441 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.695616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.7.4.129:42837)
0811 06:23:45.695768 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.695772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.696115 (+ 343us) server_negotiation.cc:407] Connection header received
0811 06:23:45.696242 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.696246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.696302 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.696379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.697318 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.697987 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.699123 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.699665 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.699777 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.703436 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.703441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.703444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.703477 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.706158 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.706792 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.706796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.706799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.706924 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.707256 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.707259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.707261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.707607 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.707816 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.708052 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.708220 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 06:23:45.712623 13922 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:23:45.711027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.7.4.129:42837)
0811 06:23:45.711228 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.711234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.711395 (+ 161us) server_negotiation.cc:407] Connection header received
0811 06:23:45.711528 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.711532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.711601 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.711676 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.712380 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34756: BlockingRecv error: recv got EOF from 127.0.0.1:34756 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 06:23:45.712908 7186 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429373903770-7186-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.714488 7186 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:45.715898 7186 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:23:45.730432 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.717312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.7.4.129:42837)
0811 06:23:45.717547 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.717552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.717829 (+ 277us) server_negotiation.cc:407] Connection header received
0811 06:23:45.718023 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.718035 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.718098 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.718211 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.719214 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.720050 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.721299 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.721753 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.721893 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.725613 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.725618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.725622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.725658 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.728183 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.728819 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.728823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.728825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.728976 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.729295 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.729298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.729300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.729554 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.729786 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.729987 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.730160 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:23:45.732110 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13482
I20260811 06:23:45.743713 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.732830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.7.4.129:42837)
0811 06:23:45.732983 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.732987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.733340 (+ 353us) server_negotiation.cc:407] Connection header received
0811 06:23:45.733468 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.733471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.733527 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.733602 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.734528 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.735186 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.736370 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.736805 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.736895 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.738868 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.738873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.738876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.738906 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.741454 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.742022 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.742026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.742028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.742161 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.742496 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.742499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.742500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.742746 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.742943 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.743197 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.743443 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:23:45.744299 13615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:45.829054 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13482
I20260811 06:23:45.850871 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13628
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:23:45.867116 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.851910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.7.4.130:33211)
0811 06:23:45.852419 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.852426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.852451 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:23:45.852509 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.852515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.852755 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.852932 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.853939 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.854947 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.856294 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.856877 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.857015 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.861015 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.861028 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.861042 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.861078 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.864019 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.864790 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.864807 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.864810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.864951 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.865366 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.865371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.865377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.865745 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.866053 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.866438 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.866731 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":242,"threads_started":1}
I20260811 06:23:45.868499 13761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:45.956759 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13628
I20260811 06:23:45.979576 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13774
Aug 11 06:23:45 dist-test-slave-07c2 krb5kdc[13369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:23:45.996069 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:45.980795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.7.4.131:41763)
0811 06:23:45.981484 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:45.981491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:45.981521 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:23:45.981591 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:45.981598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:45.981816 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:45.981946 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:45.982997 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.984111 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.985303 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:45.985870 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:45.985969 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:45.990215 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:45.990231 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:45.990247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:45.990287 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:45.993465 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.994099 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.994106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.994109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.994229 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:45.994565 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:45.994568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:45.994570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:45.994904 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:45.995162 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:45.995463 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:23:45.995703 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":372,"threads_started":1}
I20260811 06:23:45.997411 13909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:46.081266 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13774
I20260811 06:23:46.101245 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13382
I20260811 06:23:46.102392 13450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:46.207055 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13382
2026-08-11T06:23:46Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4145 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:46 dist-test-slave-07c2 krb5kdc[13942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:46 dist-test-slave-07c2 krb5kdc[13942](info): set up 2 sockets
Aug 11 06:23:46 dist-test-slave-07c2 krb5kdc[13942](info): commencing operation
krb5kdc: starting...
W20260811 06:23:47.366160 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:47 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab.
Entry for principal oryx/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab.
I20260811 06:23:47.604905 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:35353
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab
--principal=oryx/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:35353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:47.927814 13959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:47.928089 13959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:47.928257 13959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:47.935499 13959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:47.935623 13959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:47.935734 13959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:47.935846 13959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:47.935938 13959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:47.945614 13959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:35353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab
--principal=oryx/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:35353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:47.946877 13959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:47.948253 13959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:47.959939 13964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:47.959888 13965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:47.959998 13967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:47.960736 13959 server_base.cc:1034] running on GCE node
I20260811 06:23:47.961483 13959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:47.962800 13959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:47.964028 13959 hybrid_clock.cc:648] HybridClock initialized: now 1786429427963967 us; error 41 us; skew 500 ppm
I20260811 06:23:47.964357 13959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:47 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:47.967394 13959 init.cc:377] Logged in from keytab as oryx/127.7.4.190@KRBTEST.COM (short username oryx)
I20260811 06:23:47.968683 13959 webserver.cc:460] Webserver started at http://127.7.4.190:39001/ using document root <none> and password file <none>
I20260811 06:23:47.969287 13959 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:47.969388 13959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:47.969633 13959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:47.972386 13959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "a86a659668f8494db15ce16bc2d7a1c4"
format_stamp: "Formatted at 2026-08-11 06:23:47 on dist-test-slave-07c2"
server_key: "accf26dc263199980f832c8dda10e442"
server_key_iv: "6b089114f022614c2fb15b3e0fdcc6e4"
server_key_version: "encryptionkey@0"
I20260811 06:23:47.973083 13959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "a86a659668f8494db15ce16bc2d7a1c4"
format_stamp: "Formatted at 2026-08-11 06:23:47 on dist-test-slave-07c2"
server_key: "accf26dc263199980f832c8dda10e442"
server_key_iv: "6b089114f022614c2fb15b3e0fdcc6e4"
server_key_version: "encryptionkey@0"
I20260811 06:23:47.977912 13959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:23:47.981014 13975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:47.982261 13959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:23:47.982435 13959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "a86a659668f8494db15ce16bc2d7a1c4"
format_stamp: "Formatted at 2026-08-11 06:23:47 on dist-test-slave-07c2"
server_key: "accf26dc263199980f832c8dda10e442"
server_key_iv: "6b089114f022614c2fb15b3e0fdcc6e4"
server_key_version: "encryptionkey@0"
I20260811 06:23:47.982617 13959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:48.011478 13959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:48.015648 13959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:48.015967 13959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:48.035414 13959 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:35353
I20260811 06:23:48.035455 14031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:35353 every 8 connection(s)
I20260811 06:23:48.036703 13959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:23:48.040135 14032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:48.045372 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 13959
I20260811 06:23:48.045600 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:23:48.046227 7186 external_mini_cluster.cc:1428] Setting key 86e50cf60c1bb3b225a906a7f03ace68
I20260811 06:23:48.051203 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Bootstrap starting.
I20260811 06:23:48.054746 14032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:48.055914 14032 log.cc:824] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:48.059351 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: No bootstrap required, opened a new log
I20260811 06:23:48.068913 14032 raft_consensus.cc:348] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } }
Aug 11 06:23:48 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:48.069280 14032 raft_consensus.cc:374] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:48.069394 14032 raft_consensus.cc:729] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86a659668f8494db15ce16bc2d7a1c4, State: Initialized, Role: FOLLOWER
I20260811 06:23:48.070078 14032 consensus_queue.cc:260] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } }
I20260811 06:23:48.070387 14032 raft_consensus.cc:388] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:23:48.070560 14032 raft_consensus.cc:482] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:23:48.070766 14032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:48.073465 14032 raft_consensus.cc:504] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } }
I20260811 06:23:48.073990 14032 leader_election.cc:302] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a86a659668f8494db15ce16bc2d7a1c4; no voters:
I20260811 06:23:48.074725 14032 leader_election.cc:288] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:23:48.074863 14037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:48.076731 14032 sys_catalog.cc:564] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:48.079856 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:48.052046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.7.4.190:35353)
0811 06:23:48.053120 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:48.053141 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:48.055955 (+ 2814us) server_negotiation.cc:407] Connection header received
0811 06:23:48.058353 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:48.058378 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:48.058838 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:48.059217 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:48.061621 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.063748 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.064975 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.066077 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.066241 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:48.070557 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:48.070565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:48.070579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:48.070618 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:48.073841 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:48.074760 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:48.074768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:48.074775 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:48.075017 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:48.075596 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:48.075605 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:48.075610 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:48.076515 (+ 905us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:48.076941 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:48.077644 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 06:23:48.078187 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":3145,"threads_started":1}
I20260811 06:23:48.081725 14037 raft_consensus.cc:686] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 LEADER]: Becoming Leader. State: Replica: a86a659668f8494db15ce16bc2d7a1c4, State: Running, Role: LEADER
I20260811 06:23:48.082504 14037 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } }
I20260811 06:23:48.085672 14038 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } }
I20260811 06:23:48.086184 14038 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:48.086061 14039 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86a659668f8494db15ce16bc2d7a1c4. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } }
I20260811 06:23:48.087590 14039 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:48.092013 14050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:48.096436 14050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:48.104799 14050 catalog_manager.cc:1347] Generated new cluster ID: 2da51b12747e41389af652dcb0bad33e
I20260811 06:23:48.104907 14050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:48.131073 14050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:23:48.132970 14050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:48.150103 14050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Generated new TSK 0
I20260811 06:23:48.151149 14050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
I20260811 06:23:48.214473 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:48.552017 14061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:48.552399 14061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:48.552613 14061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:48.560817 14061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:48.560984 14061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:48.561174 14061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:48.571619 14061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:48.573083 14061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:48.574504 14061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:48.588256 14066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:48.588330 14067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:48.588342 14069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:48.589365 14061 server_base.cc:1034] running on GCE node
I20260811 06:23:48.589999 14061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:48.590917 14061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:48.592137 14061 hybrid_clock.cc:648] HybridClock initialized: now 1786429428592093 us; error 36 us; skew 500 ppm
I20260811 06:23:48.592446 14061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:48 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429428, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:48.595638 14061 init.cc:377] Logged in from keytab as oryx/127.7.4.129@KRBTEST.COM (short username oryx)
I20260811 06:23:48.597203 14061 webserver.cc:460] Webserver started at http://127.7.4.129:36755/ using document root <none> and password file <none>
I20260811 06:23:48.597920 14061 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:48.598024 14061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:48.598284 14061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:48.601199 14061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "124c93e3f1e9482eaf775e723d857159"
format_stamp: "Formatted at 2026-08-11 06:23:48 on dist-test-slave-07c2"
server_key: "d95985a4cd90d760c113fab1a16794c8"
server_key_iv: "9c0733602f7540a3962d2ce773278a37"
server_key_version: "encryptionkey@0"
I20260811 06:23:48.601903 14061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "124c93e3f1e9482eaf775e723d857159"
format_stamp: "Formatted at 2026-08-11 06:23:48 on dist-test-slave-07c2"
server_key: "d95985a4cd90d760c113fab1a16794c8"
server_key_iv: "9c0733602f7540a3962d2ce773278a37"
server_key_version: "encryptionkey@0"
I20260811 06:23:48.606845 14061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:23:48.610248 14078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:48.611594 14061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:23:48.611769 14061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "124c93e3f1e9482eaf775e723d857159"
format_stamp: "Formatted at 2026-08-11 06:23:48 on dist-test-slave-07c2"
server_key: "d95985a4cd90d760c113fab1a16794c8"
server_key_iv: "9c0733602f7540a3962d2ce773278a37"
server_key_version: "encryptionkey@0"
I20260811 06:23:48.611992 14061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:48.628780 14061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:48.633172 14061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:48.633478 14061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:48.634424 14061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:48.635814 14061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:48.635887 14061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:48.635972 14061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:48.636015 14061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:48.670194 14061 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:37319
I20260811 06:23:48.670243 14198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:37319 every 8 connection(s)
I20260811 06:23:48.671622 14061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:48.678998 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14061
I20260811 06:23:48.679303 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:23:48.680070 7186 external_mini_cluster.cc:1428] Setting key f373af8ee7bafd4aeb39d09b8b4dbee2
Aug 11 06:23:48 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429428, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:48.694383 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:48.674223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:33295 (local address 127.7.4.190:35353)
0811 06:23:48.674695 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:48.674701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:48.675559 (+ 858us) server_negotiation.cc:407] Connection header received
0811 06:23:48.677504 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:48.677510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:48.677597 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:48.677724 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:48.679681 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.680476 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.682414 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.682834 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.682934 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:48.688089 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:48.688094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:48.688097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:48.688130 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:48.690756 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:48.691747 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:48.691751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:48.691754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:48.691880 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:48.692366 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:48.692369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:48.692371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:48.692628 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:48.692748 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:48.693633 (+ 885us) server_negotiation.cc:299] Negotiation successful
0811 06:23:48.693918 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":268,"threads_started":1}
I20260811 06:23:48.695013 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:48.674626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:33295)
0811 06:23:48.675401 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:48.675441 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:48.677215 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:48.678006 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:48.678014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:48.678577 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:48.679474 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:48.679504 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.680649 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.680654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:48.682266 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:48.682281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:48.683083 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:48.683093 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:48.683438 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:48.684269 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:48.684310 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:48.687869 (+ 3559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:48.690908 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:48.690919 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:48.690939 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:48.691618 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:48.691988 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:48.691996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:48.692002 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:48.692254 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:48.692761 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:48.692772 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:48.693213 (+ 441us) client_negotiation.cc:769] Sending connection context
0811 06:23:48.693454 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 06:23:48.693860 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":293,"threads_started":1}
I20260811 06:23:48.696724 14199 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:48.697047 14199 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:48.697798 14199 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:48.699968 13993 ts_manager.cc:194] Registered new tserver with Master: 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319)
I20260811 06:23:48.701380 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:33295
WARNING: no policy specified for oryx/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
I20260811 06:23:48.740324 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:49.077406 14207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:49.077735 14207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:49.077833 14207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:49.085186 14207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:49.085301 14207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:49.085484 14207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:49.095372 14207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:49.096681 14207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:49.098019 14207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:49.111207 14214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:49.111312 14216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:49.111579 14212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:49.112437 14207 server_base.cc:1034] running on GCE node
I20260811 06:23:49.113099 14207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:49.113965 14207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:49.115161 14207 hybrid_clock.cc:648] HybridClock initialized: now 1786429429115124 us; error 27 us; skew 500 ppm
I20260811 06:23:49.115549 14207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:49 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429429, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:49.118544 14207 init.cc:377] Logged in from keytab as oryx/127.7.4.130@KRBTEST.COM (short username oryx)
I20260811 06:23:49.119984 14207 webserver.cc:460] Webserver started at http://127.7.4.130:35691/ using document root <none> and password file <none>
I20260811 06:23:49.120643 14207 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:49.120795 14207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:49.121090 14207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:49.123979 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "64bdf66c5d574b17a662db2e895a72b7"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "ebd6864db99fe8dcfd5e1f422f928c88"
server_key_iv: "f8aa8660544ca7554ae25fbdb7f49af4"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.124727 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "64bdf66c5d574b17a662db2e895a72b7"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "ebd6864db99fe8dcfd5e1f422f928c88"
server_key_iv: "f8aa8660544ca7554ae25fbdb7f49af4"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.129666 14207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:49.133052 14224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.134294 14207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:49.134467 14207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "64bdf66c5d574b17a662db2e895a72b7"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "ebd6864db99fe8dcfd5e1f422f928c88"
server_key_iv: "f8aa8660544ca7554ae25fbdb7f49af4"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.134658 14207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:49.146247 14207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:49.149876 14207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:49.150168 14207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:49.151063 14207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:49.152328 14207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:49.152431 14207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.152575 14207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:49.152649 14207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.186488 14207 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:36963
I20260811 06:23:49.186586 14345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:36963 every 8 connection(s)
I20260811 06:23:49.187856 14207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:49.196943 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14207
I20260811 06:23:49.197150 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:23:49.197721 7186 external_mini_cluster.cc:1428] Setting key c1fcac6793b5c2f6d774356805b8a6a2
Aug 11 06:23:49 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429429, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:49.208576 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:49.190316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:37957 (local address 127.7.4.190:35353)
0811 06:23:49.190508 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:49.190512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:49.191685 (+ 1173us) server_negotiation.cc:407] Connection header received
0811 06:23:49.193486 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:49.193491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:49.193582 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:49.193713 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:49.195208 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.195973 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.198126 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.198776 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.198920 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:49.202882 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:49.202887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:49.202890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:49.202922 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:49.205574 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:49.206283 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:49.206291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:49.206294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:49.206435 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:49.206856 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:49.206862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:49.206865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:49.207162 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:49.207346 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:49.207981 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 06:23:49.208214 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:23:49.209527 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:49.190763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:37957)
0811 06:23:49.191533 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:49.191570 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:49.193225 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:49.193957 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:49.193965 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:49.194453 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:49.195026 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:49.195043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.196145 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.196149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:49.197984 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:49.197999 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.199036 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.199041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:49.199224 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:49.200026 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:49.200054 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:49.202668 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:49.205721 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:49.205732 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:49.205746 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:49.206158 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:49.206548 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:49.206554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:49.206557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:49.206740 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:49.207309 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:49.207317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:49.207635 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 06:23:49.207891 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 06:23:49.208417 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":280,"threads_started":1}
I20260811 06:23:49.211118 14347 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:49.211481 14347 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:49.212159 14347 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:49.213707 13993 ts_manager.cc:194] Registered new tserver with Master: 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:49.214526 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:37957
WARNING: no policy specified for oryx/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
I20260811 06:23:49.255756 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:49.594765 14352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:49.595047 14352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:49.595144 14352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:49.602906 14352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:49.603003 14352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:49.603145 14352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:49.612789 14352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:35353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:49.614066 14352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:49.615475 14352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:49.628829 14357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:49.628917 14360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:49.628861 14358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:49.630017 14352 server_base.cc:1034] running on GCE node
I20260811 06:23:49.630625 14352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:49.631508 14352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:49.632716 14352 hybrid_clock.cc:648] HybridClock initialized: now 1786429429632650 us; error 59 us; skew 500 ppm
I20260811 06:23:49.633020 14352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:49 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429429, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:49.644779 14352 init.cc:377] Logged in from keytab as oryx/127.7.4.131@KRBTEST.COM (short username oryx)
I20260811 06:23:49.646248 14352 webserver.cc:460] Webserver started at http://127.7.4.131:40947/ using document root <none> and password file <none>
I20260811 06:23:49.646967 14352 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:49.647118 14352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:49.647511 14352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:49.650385 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "b84676b31ab348beb6591e1c6c605573"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "6365a0876ffd9b8bf4b4e8a18d7f0350"
server_key_iv: "d3859b7842dcccc12a73f23f022b25bc"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.651167 14352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "b84676b31ab348beb6591e1c6c605573"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "6365a0876ffd9b8bf4b4e8a18d7f0350"
server_key_iv: "d3859b7842dcccc12a73f23f022b25bc"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.656319 14352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:23:49.659747 14367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.660956 14352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:23:49.661140 14352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "b84676b31ab348beb6591e1c6c605573"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "6365a0876ffd9b8bf4b4e8a18d7f0350"
server_key_iv: "d3859b7842dcccc12a73f23f022b25bc"
server_key_version: "encryptionkey@0"
I20260811 06:23:49.661337 14352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:49.677023 14352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:49.680943 14352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:49.681247 14352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:49.682192 14352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:49.683554 14352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:49.683660 14352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.683799 14352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:49.683881 14352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:49.704416 14199 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
I20260811 06:23:49.718200 14352 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46579
I20260811 06:23:49.718245 14484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46579 every 8 connection(s)
I20260811 06:23:49.719592 14352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:49.719883 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14352
I20260811 06:23:49.720114 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:23:49.720711 7186 external_mini_cluster.cc:1428] Setting key 494f8aad45d7b1a1de9ec28ba755297a
Aug 11 06:23:49 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429429, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:49.739970 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:49.722348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:45379 (local address 127.7.4.190:35353)
0811 06:23:49.722865 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:49.722871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:49.726021 (+ 3150us) server_negotiation.cc:407] Connection header received
0811 06:23:49.726069 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:49.726072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:49.726132 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:49.726221 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:49.727725 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.728496 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.729971 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.730371 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.730460 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:49.733782 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:49.733787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:49.733791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:49.733821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:49.737002 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:49.737715 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:49.737726 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:49.737729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:49.737856 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:49.738315 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:49.738319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:49.738321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:49.738561 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:49.738715 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:49.739393 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 06:23:49.739607 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":279,"threads_started":1}
I20260811 06:23:49.740828 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:49.723056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:45379)
0811 06:23:49.723992 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:49.724051 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:49.725775 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:49.726433 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:49.726441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:49.726935 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:49.727573 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:49.727593 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.728644 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.728648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:49.729851 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:49.729863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:49.730552 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:49.730557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:49.730760 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:49.731421 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:49.731442 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:49.733565 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:49.737141 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:49.737149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:49.737164 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:49.737563 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:49.737968 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:49.737973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:49.737977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:49.738171 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:49.738690 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:49.738705 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:49.739008 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 06:23:49.739267 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:23:49.739787 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":404,"threads_started":1}
I20260811 06:23:49.742048 14485 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:49.742363 14485 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:49.743069 14485 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:49.744485 13993 ts_manager.cc:194] Registered new tserver with Master: b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579)
I20260811 06:23:49.745281 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:45379
I20260811 06:23:49.748904 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.4.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.189.keytab.
Entry for principal oryx/127.7.4.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.189.keytab.
I20260811 06:23:49.804952 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:44951
--webserver_interface=127.7.4.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.189.keytab
--principal=oryx/127.7.4.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:35353,127.7.4.189:44951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:50.132140 14492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:50.132431 14492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:50.132520 14492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:50.139667 14492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:50.139761 14492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:50.139813 14492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:50.139874 14492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:50.139917 14492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:50.149605 14492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:35353,127.7.4.189:44951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.189.keytab
--principal=oryx/127.7.4.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:44951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.4.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
W20260811 06:23:50.149952 14492 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.4.190:35353,127.7.4.189:44951'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 06:23:50.150952 14492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:50.152243 14492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:50.164004 14499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.164041 14502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.164280 14500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:50.164948 14492 server_base.cc:1034] running on GCE node
I20260811 06:23:50.165691 14492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:50.167060 14492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:50.168304 14492 hybrid_clock.cc:648] HybridClock initialized: now 1786429430168219 us; error 61 us; skew 500 ppm
I20260811 06:23:50.168639 14492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:50.171862 14492 init.cc:377] Logged in from keytab as oryx/127.7.4.189@KRBTEST.COM (short username oryx)
I20260811 06:23:50.173233 14492 webserver.cc:460] Webserver started at http://127.7.4.189:41389/ using document root <none> and password file <none>
I20260811 06:23:50.173899 14492 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:50.174082 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:50.174376 14492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:50.177361 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/instance:
uuid: "fd0fca5968034acd953d7416d6009419"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "3112632663b8ebee0853e91bbe175f98"
server_key_iv: "dd961540462c4b8733624f1cf0092450"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.178107 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal/instance:
uuid: "fd0fca5968034acd953d7416d6009419"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "3112632663b8ebee0853e91bbe175f98"
server_key_iv: "dd961540462c4b8733624f1cf0092450"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.182842 14492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:23:50.186539 14509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:50.188165 14492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:50.188359 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "fd0fca5968034acd953d7416d6009419"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "3112632663b8ebee0853e91bbe175f98"
server_key_iv: "dd961540462c4b8733624f1cf0092450"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.188606 14492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:50.214092 14492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:50.217142 14347 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
I20260811 06:23:50.218461 14492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:50.218762 14492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:50.236608 14492 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.189:44951
I20260811 06:23:50.236626 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.189:44951 every 8 connection(s)
I20260811 06:23:50.237932 14492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
I20260811 06:23:50.239532 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14492
I20260811 06:23:50.242673 14561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:50.250874 14561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.189@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:50.262969 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.241874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.7.4.189:44951)
0811 06:23:50.242645 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.242655 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.242698 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:23:50.244426 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.244446 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.244803 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.245109 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.246867 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.248287 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.250087 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.250738 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.250865 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.256036 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.256042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.256051 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.256085 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.258973 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.259725 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.259732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.259738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.259891 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.260293 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.260300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.260303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.260776 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.261106 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.261505 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.261895 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":368,"threads_started":1}
I20260811 06:23:50.267905 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.253897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45922)
0811 06:23:50.254791 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.254837 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.254996 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.255413 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.255421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.256149 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.256435 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.256444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.257442 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.257447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.258552 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.258564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.259342 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.259348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.259507 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.260080 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.260100 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.262229 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.265595 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.265605 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.265613 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.266140 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.266477 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.266480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.266482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.266597 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.267101 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.267105 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.267224 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.267413 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.267635 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":507,"threads_started":1}
I20260811 06:23:50.267905 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.253886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45922 (local address 127.7.4.190:35353)
0811 06:23:50.254456 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.254463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.254975 (+ 512us) server_negotiation.cc:407] Connection header received
0811 06:23:50.255130 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.255133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.255195 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.255327 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.256589 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.257301 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.258705 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.259113 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.259279 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.262420 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.262429 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.262434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.262481 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.265460 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.266261 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.266265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.266267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.266386 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.266681 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.266684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.266686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.266992 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.267102 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.267488 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.267666 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":260,"threads_started":1}
I20260811 06:23:50.269474 14561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.189@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
I20260811 06:23:50.280308 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.270098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43924)
0811 06:23:50.270286 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.270305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.270390 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.270626 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.270628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.270910 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.271024 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.271031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.271920 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.271923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.272755 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.272766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.273385 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.273389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.273503 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.273945 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.273960 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.275744 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.278426 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.278430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.278432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.278818 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.279111 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.279114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.279116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.279233 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.279680 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.279684 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.279808 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.279911 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.280091 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:23:50.280519 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.270164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.7.4.189:44951)
0811 06:23:50.270320 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.270324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.270342 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:23:50.270473 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.270477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.270523 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.270591 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.271132 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.271816 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.272881 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.273262 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.273349 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.275893 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.275897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.275899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.275925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.278308 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.278910 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.278913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.278915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.279022 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.279357 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.279360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.279362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.279587 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.279689 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.279986 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.280206 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":65}
I20260811 06:23:50.284725 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Bootstrap starting.
I20260811 06:23:50.287976 14561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:50.288962 14561 log.cc:824] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:50.291395 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: No bootstrap required, opened a new log
I20260811 06:23:50.298174 14561 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } }
I20260811 06:23:50.298475 14561 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:50.298592 14561 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd0fca5968034acd953d7416d6009419, State: Initialized, Role: FOLLOWER
I20260811 06:23:50.299111 14561 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } }
I20260811 06:23:50.299916 14569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } }
I20260811 06:23:50.300236 14569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:50.300818 14561 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:23:50.309144 14580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:23:50.309275 14580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:23:50.312214 14492 master_runner.cc:428] Detected that this master fd0fca5968034acd953d7416d6009419 is joining an existing cluster
I20260811 06:23:50.312296 14492 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.4.189:44951
I20260811 06:23:50.349479 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.316270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.7.4.189:44951)
0811 06:23:50.316499 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.316505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.332489 (+ 15984us) server_negotiation.cc:407] Connection header received
0811 06:23:50.332664 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.332669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.332738 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.332803 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.334877 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.335995 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.338295 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.338998 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.339091 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.342157 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.342163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.342167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.342204 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.346230 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.347015 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.347021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.347027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.347196 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.347719 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.347726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.347729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.348132 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.348276 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.348804 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.349019 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:23:50.350692 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.316866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43932)
0811 06:23:50.332401 (+ 15535us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.332418 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.332560 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.332988 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.332993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.334048 (+ 1055us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.334726 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.334740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.336232 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.336238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.338152 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.338164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.339425 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.339436 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.339677 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.340543 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.340575 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.341970 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.346362 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.346366 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.346370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.346907 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.347366 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.347371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.347377 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.347571 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.348351 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.348356 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.348527 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.349768 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.350112 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15398,"mutex_wait_us":205,"spinlock_wait_cycles":2304,"thread_start_us":15193,"threads_started":1}
I20260811 06:23:50.352802 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.316273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.7.4.190:35353)
0811 06:23:50.316592 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.316597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.332000 (+ 15403us) server_negotiation.cc:407] Connection header received
0811 06:23:50.332588 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.332597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.332697 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.332771 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.334901 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.335994 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.338313 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.338995 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.339091 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.342176 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.342182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.342187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.342243 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.346054 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.347902 (+ 1848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.347907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.347910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.348150 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.348711 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.348716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.348719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.351416 (+ 2697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.351544 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.352228 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.352489 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260811 06:23:50.353607 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.316461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45936)
0811 06:23:50.331857 (+ 15396us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.331885 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.332418 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.332988 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.332993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.334048 (+ 1055us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.334737 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.334746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.336232 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.336238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.338149 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.338164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.339447 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.339453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.339681 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.340624 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.340643 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.341970 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.346217 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.346222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.346226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.346779 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.348351 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.348356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.348360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.348560 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.351668 (+ 3108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.351672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.351866 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.352982 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.353240 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15194,"mutex_wait_us":256,"spinlock_wait_cycles":64000,"thread_start_us":15044,"threads_started":1}
WARNING: no policy specified for oryx/127.7.4.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.188@KRBTEST.COM" created.
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.190@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
I20260811 06:23:50.390563 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.368873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43946)
0811 06:23:50.369481 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.369525 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.369724 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.370454 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.370461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.371027 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.371241 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.371251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.372720 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.372726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.373779 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.373791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.374535 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.374542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.374758 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.375461 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.375483 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.382075 (+ 6592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.387010 (+ 4935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.387022 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.387032 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.387731 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.388197 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.388204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.388207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.388424 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.389197 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.389205 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.389434 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.389670 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.390071 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":237,"threads_started":1}
I20260811 06:23:50.391820 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.368901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.7.4.189:44951)
0811 06:23:50.369090 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.369098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.369640 (+ 542us) server_negotiation.cc:407] Connection header received
0811 06:23:50.369949 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.369957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.370063 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.370177 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.371411 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.372561 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.373915 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.374367 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.374516 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.382306 (+ 7790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.382314 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.382320 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.382364 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.386829 (+ 4465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.387891 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.387898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.387903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.388091 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.388558 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.388563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.388566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.388956 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.389106 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.391114 (+ 2008us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.391432 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:23:50.394158 13993 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.4.189:44951: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fd0fca5968034acd953d7416d6009419"
member_type: NON_VOTER
last_known_addr {
host: "127.7.4.189"
port: 44951
}
attrs {
promote: true
}
}
dest_uuid: "a86a659668f8494db15ce16bc2d7a1c4"
cas_config_opid_index: -1
I20260811 06:23:50.395903 13993 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.188.keytab.
Entry for principal oryx/127.7.4.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.188.keytab.
I20260811 06:23:50.399057 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:39139
--webserver_interface=127.7.4.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.188.keytab
--principal=oryx/127.7.4.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:23:50.407719 14536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:50.407354 14595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fd0fca5968034acd953d7416d6009419 (127.7.4.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.411356 14536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 06:23:50.412819 14594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:50.431417 14595 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.4.189:44951
I20260811 06:23:50.434526 14492 master.cc:537] Master@127.7.4.189:44951 shutting down...
I20260811 06:23:50.434405 14594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.434747 14594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
W20260811 06:23:50.470702 14515 tablet.cc:2369] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:50.481956 14536 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fd0fca5968034acd953d7416d6009419 (127.7.4.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.491871 14605 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.492313 14605 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:50.492384 14569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86a659668f8494db15ce16bc2d7a1c4. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.492619 14569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:50.492766 14602 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.493140 14602 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:50.496207 14492 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 06:23:50.496676 14492 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 06:23:50.496848 14492 tablet_replica.cc:330] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: stopping tablet replica
I20260811 06:23:50.514946 14492 master.cc:552] Master@127.7.4.189:44951 shutdown complete.
I20260811 06:23:50.515105 14492 master_runner.cc:305] Clearing existing system tablet
I20260811 06:23:50.515811 14492 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:50.515951 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:50.519838 14492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:23:50.521567 14607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:50.521908 14492 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:50.522179 14492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 06:23:50.522266 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "fd0fca5968034acd953d7416d6009419"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "3112632663b8ebee0853e91bbe175f98"
server_key_iv: "dd961540462c4b8733624f1cf0092450"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.527335 14492 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:50.534624 14492 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:23:50.534852 14492 log.cc:1196] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 06:23:50.535434 14492 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Deleting consensus metadata
I20260811 06:23:50.535722 14492 master_runner.cc:315] Copying system tablet from 127.7.4.190:35353
I20260811 06:23:50.539553 14492 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: tablet copy: Beginning tablet copy session from remote peer at address 127.7.4.190:35353
I20260811 06:23:50.556336 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.540635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.7.4.190:35353)
0811 06:23:50.540821 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.540827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.541377 (+ 550us) server_negotiation.cc:407] Connection header received
0811 06:23:50.541569 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.541575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.541672 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.541776 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:50.542855 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.544026 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.545964 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.546613 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.546742 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.549171 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:50.549178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:50.549183 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:50.549224 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:50.552957 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.553861 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.553868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.553871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.554034 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:50.554468 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:50.554473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:50.554477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:50.554868 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:50.555024 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.555688 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.555926 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:23:50.556391 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.540548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45938)
0811 06:23:50.541235 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.541256 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.541428 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.541832 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.541837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.542405 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:50.542678 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.542691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.544220 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.544228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.545794 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.545813 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.546807 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.546813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.547005 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.547770 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:50.547793 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:50.548930 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:50.553112 (+ 4182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.553118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.553122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.553712 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.554172 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:50.554177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:50.554180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:50.554344 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:50.555022 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:50.555028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:50.555247 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.555480 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.555824 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":373,"threads_started":1}
I20260811 06:23:50.557449 14015 tablet_copy_service.cc:140] P a86a659668f8494db15ce16bc2d7a1c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fd0fca5968034acd953d7416d6009419 ({username='oryx', principal='oryx/127.7.4.189@KRBTEST.COM'} at 127.0.0.1:45938)
I20260811 06:23:50.557934 14015 tablet_copy_service.cc:161] P a86a659668f8494db15ce16bc2d7a1c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fd0fca5968034acd953d7416d6009419 at {username='oryx', principal='oryx/127.7.4.189@KRBTEST.COM'} at 127.0.0.1:45938: session id = fd0fca5968034acd953d7416d6009419-00000000000000000000000000000000
I20260811 06:23:50.562985 14015 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:23:50.565863 14492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:23:50.572813 14492 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: tablet copy: Starting download of 0 data blocks...
I20260811 06:23:50.573197 14492 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: tablet copy: Starting download of 1 WAL segments...
I20260811 06:23:50.576785 14492 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:23:50.581700 14015 tablet_copy_service.cc:342] P a86a659668f8494db15ce16bc2d7a1c4: Request end of tablet copy session fd0fca5968034acd953d7416d6009419-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.4.189@KRBTEST.COM'} at 127.0.0.1:45938
I20260811 06:23:50.581948 14015 tablet_copy_service.cc:433] P a86a659668f8494db15ce16bc2d7a1c4: ending tablet copy session fd0fca5968034acd953d7416d6009419-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fd0fca5968034acd953d7416d6009419
I20260811 06:23:50.594978 14492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:50.597757 14615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.600756 14616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.611356 14619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:50.617223 14492 server_base.cc:1034] running on GCE node
I20260811 06:23:50.617754 14492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:50.618677 14492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:50.620044 14492 hybrid_clock.cc:648] HybridClock initialized: now 1786429430619942 us; error 40 us; skew 500 ppm
I20260811 06:23:50.620512 14492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:50.623314 14492 init.cc:377] Logged in from keytab as oryx/127.7.4.189@KRBTEST.COM (short username oryx)
I20260811 06:23:50.625046 14492 webserver.cc:460] Webserver started at http://127.7.4.189:33915/ using document root <none> and password file <none>
I20260811 06:23:50.625469 14492 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:50.625574 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:50.629354 14492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:23:50.631839 14626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:50.632411 14492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 06:23:50.632588 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "fd0fca5968034acd953d7416d6009419"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "3112632663b8ebee0853e91bbe175f98"
server_key_iv: "dd961540462c4b8733624f1cf0092450"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.632781 14492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:50.666127 14492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:50.672087 14492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:50.746181 14492 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.189:44951
I20260811 06:23:50.746214 14680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.189:44951 every 8 connection(s)
I20260811 06:23:50.747785 14485 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
I20260811 06:23:50.747833 14492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
I20260811 06:23:50.758152 14681 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:23:50.762380 14681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Bootstrap starting.
I20260811 06:23:50.778246 14681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:23:50.778798 14681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: Bootstrap complete.
I20260811 06:23:50.779902 14681 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } }
I20260811 06:23:50.780181 14681 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fd0fca5968034acd953d7416d6009419, State: Initialized, Role: LEARNER
I20260811 06:23:50.780511 14681 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } }
I20260811 06:23:50.781662 14681 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:50.781483 14684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.782051 14684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:50.796788 14695 catalog_manager.cc:1259] Loaded cluster ID: 2da51b12747e41389af652dcb0bad33e
I20260811 06:23:50.796890 14695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: loading cluster ID for follower catalog manager: success
I20260811 06:23:50.801211 14695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: acquiring CA information for follower catalog manager: success
I20260811 06:23:50.804216 14695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:23:50.868643 14596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:50.868911 14596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:50.869000 14596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:50.876080 14596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:23:50.876186 14596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:50.876233 14596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:50.876330 14596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:23:50.876380 14596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:23:50.887508 14596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.188.keytab
--principal=oryx/127.7.4.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:39139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.4.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:50.888773 14596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:50.890017 14596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:50.901722 14701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.901747 14702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:50.902513 14704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:50.903069 14596 server_base.cc:1034] running on GCE node
I20260811 06:23:50.903891 14596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:50.905200 14596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:50.906459 14596 hybrid_clock.cc:648] HybridClock initialized: now 1786429430906403 us; error 52 us; skew 500 ppm
I20260811 06:23:50.906805 14596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:50 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:50.909907 14595 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.910212 14595 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:50.910490 14596 init.cc:377] Logged in from keytab as oryx/127.7.4.188@KRBTEST.COM (short username oryx)
I20260811 06:23:50.913116 14596 webserver.cc:460] Webserver started at http://127.7.4.188:35417/ using document root <none> and password file <none>
I20260811 06:23:50.913975 14596 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:50.914156 14596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:50.914528 14596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:23:50.918753 14596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/instance:
uuid: "68891d8213204e0591f67e622bfc2aff"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "2a82eeb39eca0a0e82410cdebe7505b1"
server_key_iv: "908d00b94fafcc77f41b8223bd682c3d"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.919541 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.910350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.7.4.189:44951)
0811 06:23:50.912932 (+ 2582us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.912937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.912952 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:23:50.913013 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:50.913017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:50.913101 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:50.913194 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:50.913772 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.914973 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.917554 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.918352 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.918454 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.918608 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:50.919020 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 06:23:50.919118 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2463,"thread_start_us":2339,"threads_started":1}
I20260811 06:23:50.919616 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.910184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43948)
0811 06:23:50.911018 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:50.911041 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:50.912400 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:50.913298 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:50.913303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:50.913335 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:50.913597 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.913612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.915134 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.915143 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.917388 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:50.917406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:50.918531 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:50.918540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:50.918808 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:50.918871 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 06:23:50.918983 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 06:23:50.919136 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":522,"threads_started":1}
I20260811 06:23:50.919853 14596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal/instance:
uuid: "68891d8213204e0591f67e622bfc2aff"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "2a82eeb39eca0a0e82410cdebe7505b1"
server_key_iv: "908d00b94fafcc77f41b8223bd682c3d"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.920706 14684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86a659668f8494db15ce16bc2d7a1c4. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:50.920986 14684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:50.921733 14595 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: attempting to promote NON_VOTER fd0fca5968034acd953d7416d6009419 to VOTER
I20260811 06:23:50.922538 14595 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } }
I20260811 06:23:50.924144 14654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 LEARNER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 06:23:50.924817 14594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 06:23:50.926137 14596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 06:23:50.927848 14595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, fd0fca5968034acd953d7416d6009419 (127.7.4.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } }
I20260811 06:23:50.928494 14654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, fd0fca5968034acd953d7416d6009419 (127.7.4.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } }
I20260811 06:23:50.930791 14595 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } }
I20260811 06:23:50.931053 14595 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:50.930910 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } }
I20260811 06:23:50.931444 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:50.932618 14684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } }
I20260811 06:23:50.932788 14714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:50.932860 14684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:50.934269 14596 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 06:23:50.934469 14596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "68891d8213204e0591f67e622bfc2aff"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "2a82eeb39eca0a0e82410cdebe7505b1"
server_key_iv: "908d00b94fafcc77f41b8223bd682c3d"
server_key_version: "encryptionkey@0"
I20260811 06:23:50.934712 14596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:50.959987 14596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:50.963982 14596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:50.964247 14596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:50.985050 14596 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.188:39139
I20260811 06:23:50.985069 14767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.188:39139 every 8 connection(s)
I20260811 06:23:50.986804 14596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
I20260811 06:23:50.990571 14768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:50.997007 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14596
I20260811 06:23:50.996699 14768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.188@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
I20260811 06:23:51.018474 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.000269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.7.4.190:35353)
0811 06:23:51.000454 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.000460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.001078 (+ 618us) server_negotiation.cc:407] Connection header received
0811 06:23:51.002032 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.002038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.002114 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.002221 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.003426 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.004581 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.006803 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.007552 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.007715 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.011591 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.011598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.011603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.011647 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.014903 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.015925 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.015933 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.015938 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.016097 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.016599 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.016603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.016605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.016879 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.017026 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.017972 (+ 946us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.018203 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:23:51.019579 14772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.000111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45946)
0811 06:23:51.000923 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.000968 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.001797 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.002281 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.002286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.002875 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.003221 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.003236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.004715 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.004720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.006671 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.006681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.007769 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.007774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.007962 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.008700 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.008730 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.011400 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.015028 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.015042 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.015055 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.015786 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.016221 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.016228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.016232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.016481 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.017093 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.017102 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.017551 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.017823 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.018239 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":403,"threads_started":1}
I20260811 06:23:51.020514 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:50.999091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.7.4.188:39139)
0811 06:23:50.999806 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:50.999816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:50.999869 (+ 53us) server_negotiation.cc:407] Connection header received
0811 06:23:51.001786 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.001805 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.002061 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.002254 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.004437 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.006322 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.008130 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.008935 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.009069 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.013724 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.013733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.013745 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.013779 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.017448 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.018209 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.018216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.018220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.018378 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.018843 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.018847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.018849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.019176 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.019501 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.019830 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.020067 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":355,"threads_started":1}
I20260811 06:23:51.021306 14768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.188@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
I20260811 06:23:51.034683 14772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.022141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43962)
0811 06:23:51.022347 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.022366 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.022483 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.022819 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.022823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.023344 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.023519 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.023530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.024621 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.024625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.025679 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.025695 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.026683 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.026687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.026834 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.027331 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.027360 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.029272 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.032354 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.032357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.032360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.032787 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.033107 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.033110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.033112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.033265 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.033853 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.033856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.034008 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.034118 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.034384 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:23:51.034730 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.022164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.7.4.189:44951)
0811 06:23:51.022396 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.022400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.022418 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:23:51.022616 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.022620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.022682 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.022770 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.023683 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.024490 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.025866 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.026552 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.026646 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.029439 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.029443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.029447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.029481 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.032241 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.032882 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.032885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.032887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.033016 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.033372 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.033375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.033377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.033752 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.033861 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.034217 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.034416 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 06:23:51.035946 14768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429430, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.188@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
I20260811 06:23:51.048243 14772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.036778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.0.0.1:34638)
0811 06:23:51.036978 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.036997 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.037090 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.037380 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.037384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.037762 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.037956 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.037974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.038973 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.038980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.039995 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.040007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.040709 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.040715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.040920 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.041430 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.041445 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.043377 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.045964 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.045971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.045975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.046305 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.046626 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.046631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.046637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.046789 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.047264 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.047269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.047414 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.047544 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.047820 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:23:51.048280 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.036850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.7.4.188:39139)
0811 06:23:51.037055 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.037060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.037076 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:23:51.037205 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.037213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.037272 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.037366 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.038120 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.038855 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.040146 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.040563 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.040695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.043544 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.043548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.043551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.043577 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.045846 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.046411 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.046418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.046421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.046525 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.046889 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.046895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.046898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.047140 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.047308 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.047622 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.047954 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":108}
I20260811 06:23:51.052949 14768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Bootstrap starting.
I20260811 06:23:51.055975 14768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:51.056885 14768 log.cc:824] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:51.059381 14768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: No bootstrap required, opened a new log
I20260811 06:23:51.066716 14768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } }
I20260811 06:23:51.066982 14768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:51.067067 14768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68891d8213204e0591f67e622bfc2aff, State: Initialized, Role: FOLLOWER
I20260811 06:23:51.067606 14768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } }
I20260811 06:23:51.068429 14774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } } }
I20260811 06:23:51.068782 14774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:51.069311 14768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:51.076020 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14061
W20260811 06:23:51.079382 14786 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:23:51.079522 14786 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:23:51.084358 14596 master_runner.cc:428] Detected that this master 68891d8213204e0591f67e622bfc2aff is joining an existing cluster
I20260811 06:23:51.084470 14596 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.4.188:39139
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.129@KRBTEST.COM
I20260811 06:23:51.104589 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.077573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.7.4.129:37319)
0811 06:23:51.080886 (+ 3313us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.080895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.080935 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:23:51.081008 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.081017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.081247 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.081408 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.083138 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.084795 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.086771 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.087667 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.087781 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.096585 (+ 8804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.096605 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.096626 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.096680 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.100769 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.101649 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.101657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.101660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.101850 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.102306 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.102311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.102315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.102808 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.103124 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.103891 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.104151 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2389,"thread_start_us":2225,"threads_started":1}
I20260811 06:23:51.106714 14194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:51.113929 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.089707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.7.4.190:35353)
0811 06:23:51.089945 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.089951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.091774 (+ 1823us) server_negotiation.cc:407] Connection header received
0811 06:23:51.092088 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.092097 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.092201 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.092331 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.094468 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.096020 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.098763 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.099681 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.099803 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.103668 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.103678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.103686 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.103736 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.108429 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.110147 (+ 1718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.110155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.110161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.110418 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.111038 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.111046 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.111051 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.111522 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.111672 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.113350 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.113553 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:23:51.114012 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.088939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45958)
0811 06:23:51.091652 (+ 2713us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.091672 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.091916 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.092607 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.092615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.093720 (+ 1105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.094243 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.094261 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.096264 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.096272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.098556 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.098577 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.100158 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.100168 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.100422 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.101317 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.101349 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.102844 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.109381 (+ 6537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.109388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.109394 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.109974 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.110610 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.110618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.110624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.110883 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.111786 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.111794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.112096 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.113238 (+ 1142us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.113515 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"thread_start_us":915,"threads_started":1}
I20260811 06:23:51.124926 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.089944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.7.4.189:44951)
0811 06:23:51.090094 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.090098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.106786 (+ 16688us) server_negotiation.cc:407] Connection header received
0811 06:23:51.107088 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.107098 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.107204 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.107347 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.109321 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.110845 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.113171 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.113790 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.113914 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.116482 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.116488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.116492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.116540 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.121442 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.122245 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.122252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.122258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.122422 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.122904 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.122911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.122916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.123272 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.123456 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.124216 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.124456 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:23:51.126065 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.090875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43966)
0811 06:23:51.106661 (+ 15786us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.106692 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.106906 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.107594 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.107601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.108704 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.109120 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.109136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.111067 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.111076 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.113027 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.113042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.114168 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.114175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.114345 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.114993 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.115023 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.116273 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.121594 (+ 5321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.121601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.121609 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.122116 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.122582 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.122587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.122589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.122778 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.123546 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.123556 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.123858 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.125122 (+ 1264us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.125499 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15597,"thread_start_us":21754,"threads_started":1}
I20260811 06:23:51.132815 14794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.091482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.0.0.1:34650)
0811 06:23:51.112955 (+ 21473us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.112969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.113097 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.114657 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.114663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.117173 (+ 2510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.117525 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.117534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.118554 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.118560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.121809 (+ 3249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.121822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.123142 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.123148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.123375 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.124056 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.124083 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.125492 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.129557 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.129565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.129571 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.130238 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.130674 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.130681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.130687 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.130829 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.131698 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.131703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.131904 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.132042 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.132316 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21351,"thread_start_us":11884,"threads_started":1}
I20260811 06:23:51.133724 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.090433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.7.4.188:39139)
0811 06:23:51.090623 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.090628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.113048 (+ 22420us) server_negotiation.cc:407] Connection header received
0811 06:23:51.114309 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.114314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.114377 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.114441 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.117660 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.118349 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.121950 (+ 3601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.122727 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.122828 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.125664 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.125676 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.125680 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.125714 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.129434 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.130372 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.130380 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.130386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.130545 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.130945 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.130950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.130953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.131463 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.131645 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.133112 (+ 1467us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.133349 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.190@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
I20260811 06:23:51.164059 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.120713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.0.0.1:34666)
0811 06:23:51.120919 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.120939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.121041 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.128701 (+ 7660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.128707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.129243 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.129455 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.129470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.149592 (+ 20122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.149600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.151078 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.151102 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.152212 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.152219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.152401 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.153272 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.153297 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.156315 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.160852 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.160858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.160862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.161439 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.161909 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.161915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.161919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.162102 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.162778 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.162784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.163118 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.163324 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.163647 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:23:51.164146 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.120617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.7.4.188:39139)
0811 06:23:51.128228 (+ 7611us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.128236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.128267 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:23:51.128370 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.128377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.128465 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.128629 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.147910 (+ 19281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.149342 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.151265 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.152008 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.152170 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.156578 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.156585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.156590 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.156630 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.160682 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.161581 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.161587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.161591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.161764 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.162234 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.162239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.162242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.162624 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.162770 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.163449 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.163699 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7449,"thread_start_us":7252,"threads_started":1}
I20260811 06:23:51.166460 13993 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.4.188:39139: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "68891d8213204e0591f67e622bfc2aff"
member_type: NON_VOTER
last_known_addr {
host: "127.7.4.188"
port: 39139
}
attrs {
promote: true
}
}
dest_uuid: "a86a659668f8494db15ce16bc2d7a1c4"
cas_config_opid_index: 6
I20260811 06:23:51.167369 13993 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } }
I20260811 06:23:51.170013 14654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:23:51.171185 14742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:51.175379 14742 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:23:51.175824 14595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 06:23:51.176829 14595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:51.179023 14598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 68891d8213204e0591f67e622bfc2aff (127.7.4.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.183873 14654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 68891d8213204e0591f67e622bfc2aff (127.7.4.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.187760 14598 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.4.188:39139
I20260811 06:23:51.188606 14596 master.cc:537] Master@127.7.4.188:39139 shutting down...
I20260811 06:23:51.189383 14595 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.189718 14595 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.192716 14594 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.192955 14594 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.193645 14684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.193930 14684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:51.196473 14742 raft_consensus.cc:858] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } }
W20260811 06:23:51.196659 14742 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } }
I20260811 06:23:51.207620 14742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fd0fca5968034acd953d7416d6009419 (127.7.4.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:51.212139 14805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:51.212531 14805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 06:23:51.212648 14774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86a659668f8494db15ce16bc2d7a1c4. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:51.212899 14774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 06:23:51.212931 14801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: NON_VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: true } } }
I20260811 06:23:51.213317 14801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 06:23:51.215946 14596 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 06:23:51.216442 14596 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 06:23:51.216395 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.216632 14596 tablet_replica.cc:330] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: stopping tablet replica
I20260811 06:23:51.216692 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
W20260811 06:23:51.216943 13976 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 06:23:51.221055 13976 consensus_peers.cc:487] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 -> Peer 68891d8213204e0591f67e622bfc2aff (127.7.4.188:39139): Couldn't send request to peer 68891d8213204e0591f67e622bfc2aff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 06:23:51.227792 14596 master.cc:552] Master@127.7.4.188:39139 shutdown complete.
I20260811 06:23:51.227941 14596 master_runner.cc:305] Clearing existing system tablet
I20260811 06:23:51.228394 14596 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:51.228569 14596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:51.231774 14596 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:23:51.233731 14808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:51.234221 14596 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:51.234552 14596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 06:23:51.234702 14596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "68891d8213204e0591f67e622bfc2aff"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "2a82eeb39eca0a0e82410cdebe7505b1"
server_key_iv: "908d00b94fafcc77f41b8223bd682c3d"
server_key_version: "encryptionkey@0"
I20260811 06:23:51.239464 14596 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:51.246685 14596 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:23:51.246948 14596 log.cc:1196] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 06:23:51.247575 14596 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Deleting consensus metadata
I20260811 06:23:51.247870 14596 master_runner.cc:315] Copying system tablet from 127.7.4.190:35353
I20260811 06:23:51.250757 14596 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: tablet copy: Beginning tablet copy session from remote peer at address 127.7.4.190:35353
I20260811 06:23:51.259505 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14061
I20260811 06:23:51.272238 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.251711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.7.4.190:35353)
0811 06:23:51.252009 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.252015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.256010 (+ 3995us) server_negotiation.cc:407] Connection header received
0811 06:23:51.256330 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.256339 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.256449 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.256611 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.258115 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.259189 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.261226 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.261903 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.262038 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.264628 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.264637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.264645 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.264691 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.268595 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.269512 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.269518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.269522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.269690 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.270183 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.270188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.270192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.270591 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.270769 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.271360 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.271651 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260811 06:23:51.273501 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.251633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:45960)
0811 06:23:51.255862 (+ 4229us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.255891 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.256138 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.256667 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.256675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.257417 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.257874 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.257890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.259390 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.259396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.261015 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.261031 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.262305 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.262311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.262521 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.263218 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.263247 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.264373 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.268754 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.268762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.268769 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.269373 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.269825 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.269832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.269838 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.270050 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.270735 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.270743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.270989 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.272525 (+ 1536us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.272899 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4053,"thread_start_us":3892,"threads_started":1}
I20260811 06:23:51.274402 14015 tablet_copy_service.cc:140] P a86a659668f8494db15ce16bc2d7a1c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 68891d8213204e0591f67e622bfc2aff ({username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:45960)
I20260811 06:23:51.274586 14015 tablet_copy_service.cc:161] P a86a659668f8494db15ce16bc2d7a1c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 68891d8213204e0591f67e622bfc2aff at {username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:45960: session id = 68891d8213204e0591f67e622bfc2aff-00000000000000000000000000000000
I20260811 06:23:51.278208 14015 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:23:51.280483 14596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:23:51.286381 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:37319
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:35353
--webserver_port=36755
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:23:51.287894 14596 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: tablet copy: Starting download of 0 data blocks...
I20260811 06:23:51.288223 14596 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: tablet copy: Starting download of 1 WAL segments...
I20260811 06:23:51.290504 14596 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:23:51.293890 14015 tablet_copy_service.cc:342] P a86a659668f8494db15ce16bc2d7a1c4: Request end of tablet copy session 68891d8213204e0591f67e622bfc2aff-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:45960
I20260811 06:23:51.294039 14015 tablet_copy_service.cc:433] P a86a659668f8494db15ce16bc2d7a1c4: ending tablet copy session 68891d8213204e0591f67e622bfc2aff-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 68891d8213204e0591f67e622bfc2aff
I20260811 06:23:51.306337 14596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:51.309427 14817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:51.310626 14818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:51.317673 14820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:51.318120 14596 server_base.cc:1034] running on GCE node
I20260811 06:23:51.318598 14596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:51.319442 14596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:51.320704 14596 hybrid_clock.cc:648] HybridClock initialized: now 1786429431320636 us; error 52 us; skew 500 ppm
I20260811 06:23:51.321110 14596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:51.323668 14596 init.cc:377] Logged in from keytab as oryx/127.7.4.188@KRBTEST.COM (short username oryx)
I20260811 06:23:51.328398 14596 webserver.cc:460] Webserver started at http://127.7.4.188:36489/ using document root <none> and password file <none>
I20260811 06:23:51.328819 14596 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:51.328970 14596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:51.332186 14596 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:23:51.334379 14827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:51.334823 14596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 06:23:51.334985 14596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "68891d8213204e0591f67e622bfc2aff"
format_stamp: "Formatted at 2026-08-11 06:23:50 on dist-test-slave-07c2"
server_key: "2a82eeb39eca0a0e82410cdebe7505b1"
server_key_iv: "908d00b94fafcc77f41b8223bd682c3d"
server_key_version: "encryptionkey@0"
I20260811 06:23:51.335206 14596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:51.368602 14596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:51.373140 14596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:51.400341 14596 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.188:39139
I20260811 06:23:51.400400 14883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.188:39139 every 8 connection(s)
I20260811 06:23:51.401599 14596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
I20260811 06:23:51.407401 14885 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:23:51.411420 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Bootstrap starting.
I20260811 06:23:51.427771 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:23:51.428442 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Bootstrap complete.
I20260811 06:23:51.429694 14885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } }
I20260811 06:23:51.429994 14885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 68891d8213204e0591f67e622bfc2aff, State: Initialized, Role: LEARNER
I20260811 06:23:51.430415 14885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } }
I20260811 06:23:51.431584 14889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.432016 14889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:51.432071 14885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:23:51.448427 14902 catalog_manager.cc:1259] Loaded cluster ID: 2da51b12747e41389af652dcb0bad33e
I20260811 06:23:51.448590 14902 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: loading cluster ID for follower catalog manager: success
I20260811 06:23:51.452553 14902 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: acquiring CA information for follower catalog manager: success
I20260811 06:23:51.454977 14902 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:23:51.698104 14816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:51.698365 14816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:51.698585 14816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:51.706121 14816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:51.706283 14816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:51.706472 14816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:23:51.717674 14816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:37319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=36755
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:51.719281 14816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:51.720724 14816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:51.737576 14908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:51.738063 14907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:51.738711 14910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:51.738960 14816 server_base.cc:1034] running on GCE node
I20260811 06:23:51.739683 14816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:51.740595 14816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:51.741832 14816 hybrid_clock.cc:648] HybridClock initialized: now 1786429431741777 us; error 49 us; skew 500 ppm
I20260811 06:23:51.742177 14816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:51.752552 14816 init.cc:377] Logged in from keytab as oryx/127.7.4.129@KRBTEST.COM (short username oryx)
I20260811 06:23:51.753489 14913 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.753815 14913 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.760629 14816 webserver.cc:460] Webserver started at http://127.7.4.129:36755/ using document root <none> and password file <none>
I20260811 06:23:51.761482 14816 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:51.761540 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.744777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.7.4.188:39139)
0811 06:23:51.751522 (+ 6745us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.751529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.752203 (+ 674us) server_negotiation.cc:407] Connection header received
0811 06:23:51.752286 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.752289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.752387 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.752528 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:51.753134 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.755110 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.758651 (+ 3541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.760308 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.760442 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.760732 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.761091 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.761256 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6600,"thread_start_us":7090,"threads_started":1}
I20260811 06:23:51.761623 14816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:51.762218 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.744677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.0.0.1:34672)
0811 06:23:51.751912 (+ 7235us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.751936 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.752072 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.752547 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.752560 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.752588 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:51.752904 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.752922 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.755536 (+ 2614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.755545 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.758390 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.758412 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.760658 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.760664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.760860 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.760897 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.761763 (+ 866us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.761898 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7126,"thread_start_us":7002,"threads_started":1}
I20260811 06:23:51.763195 14858 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260811 06:23:51.763655 14889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86a659668f8494db15ce16bc2d7a1c4. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: NON_VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: true } } }
I20260811 06:23:51.763934 14889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: LEARNER
I20260811 06:23:51.764348 14913 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: attempting to promote NON_VOTER 68891d8213204e0591f67e622bfc2aff to VOTER
I20260811 06:23:51.765113 14913 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:51.767167 14858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 LEARNER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:23:51.767246 14654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Refusing update from remote peer a86a659668f8494db15ce16bc2d7a1c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:23:51.767870 14913 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:23:51.768262 14915 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:23:51.771338 14914 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68891d8213204e0591f67e622bfc2aff (127.7.4.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.771971 14816 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.005s
I20260811 06:23:51.771832 14654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68891d8213204e0591f67e622bfc2aff (127.7.4.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.775686 14914 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.776049 14914 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.776314 14914 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.776770 14914 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.777609 14927 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.777988 14927 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:51.778479 14926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:51.780112 14816 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.007s sys 0.000s
I20260811 06:23:51.780898 14914 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.781136 14914 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4 [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:51.780313 14816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "124c93e3f1e9482eaf775e723d857159"
format_stamp: "Formatted at 2026-08-11 06:23:48 on dist-test-slave-07c2"
server_key: "d95985a4cd90d760c113fab1a16794c8"
server_key_iv: "9c0733602f7540a3962d2ce773278a37"
server_key_version: "encryptionkey@0"
I20260811 06:23:51.781399 14858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 68891d8213204e0591f67e622bfc2aff (127.7.4.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.781731 14816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:51.784773 14889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a86a659668f8494db15ce16bc2d7a1c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:51.785050 14889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:51.799471 14816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:51.803093 14816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:51.803385 14816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:51.804378 14816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:51.805783 14816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:51.805894 14816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:51.806028 14816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:51.806128 14816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:51.846696 14816 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:37319
I20260811 06:23:51.846822 15056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:37319 every 8 connection(s)
I20260811 06:23:51.848237 14816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:23:51.855396 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 14816
I20260811 06:23:51.855813 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14207
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.130@KRBTEST.COM
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
Aug 11 06:23:51 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429431, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
I20260811 06:23:51.909950 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.852003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45921 (local address 127.7.4.189:44951)
0811 06:23:51.877779 (+ 25776us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.877807 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.884951 (+ 7144us) server_negotiation.cc:407] Connection header received
0811 06:23:51.888296 (+ 3345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.888302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.888394 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.888502 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.891548 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892688 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.895609 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896276 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896380 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.901779 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.901784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.901787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.901859 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.905958 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.906953 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.906958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.906961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.907115 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.907771 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.907778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.907781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.908157 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.908306 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.909309 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.909530 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25555,"thread_start_us":25329,"threads_started":1}
I20260811 06:23:51.910758 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.851440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:35719 (local address 127.7.4.190:35353)
0811 06:23:51.880157 (+ 28717us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.880166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.885247 (+ 5081us) server_negotiation.cc:407] Connection header received
0811 06:23:51.888369 (+ 3122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.888374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.888458 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.888577 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.891551 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892869 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.895750 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896469 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896601 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.902669 (+ 6068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.902676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.902681 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.902736 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.907271 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908275 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.908282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.908285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.908433 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908996 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.909001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.909004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.909352 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.909487 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.910063 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.910277 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28520,"thread_start_us":28708,"threads_started":1}
I20260811 06:23:51.910772 15069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.856880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.7.4.130:36963)
0811 06:23:51.883268 (+ 26388us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.883275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.884950 (+ 1675us) server_negotiation.cc:407] Connection header received
0811 06:23:51.885252 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.885282 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.885553 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.885692 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.888021 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.889699 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.891747 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892596 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.892720 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.899572 (+ 6852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.899585 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.899599 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.899644 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.906382 (+ 6738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.907336 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.907348 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.907353 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.907521 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908011 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.908017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.908021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.908510 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.908866 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.910054 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.910305 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26180,"thread_start_us":26011,"threads_started":1}
I20260811 06:23:51.911921 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.852536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:41855 (local address 127.7.4.188:39139)
0811 06:23:51.852808 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:51.852817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:51.885547 (+ 32730us) server_negotiation.cc:407] Connection header received
0811 06:23:51.888517 (+ 2970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:51.888523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:51.888671 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:51.888754 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:51.891546 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892622 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.895619 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896348 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896430 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.903511 (+ 7081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:51.903518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:51.903522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:51.903562 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:51.907406 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908318 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.908326 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.908329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.908500 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:51.909028 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:51.909033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:51.909036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:51.909391 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:51.909521 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:51.911231 (+ 1710us) server_negotiation.cc:299] Negotiation successful
0811 06:23:51.911546 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 06:23:51.912200 14341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:51.912197 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.852849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.7.4.129:45921)
0811 06:23:51.884714 (+ 31865us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.884782 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.887969 (+ 3187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.889073 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.889085 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.890131 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.891342 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.891363 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892887 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.892893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.895433 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.895442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896811 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.897375 (+ 556us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.898257 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.898304 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.901516 (+ 3212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.906164 (+ 4648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.906172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.906190 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.906804 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.907273 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.907278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.907281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.907604 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.908400 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.908410 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.908854 (+ 444us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.910185 (+ 1331us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.910690 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31716,"mutex_wait_us":523,"spinlock_wait_cycles":29568,"thread_start_us":31954,"threads_started":1}
I20260811 06:23:51.912225 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.851968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:35719)
0811 06:23:51.885045 (+ 33077us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.885064 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.887973 (+ 2909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.889074 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.889085 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.890175 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.891333 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.891360 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.893070 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.893075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.895556 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.895572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896929 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.897396 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.898369 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.898400 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.902441 (+ 4041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.907440 (+ 4999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.907447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.907452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.908130 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.908612 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.908620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.908848 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.909581 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.909586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.909762 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.911021 (+ 1259us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.911331 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31879,"mutex_wait_us":304,"spinlock_wait_cycles":27136,"thread_start_us":32017,"threads_started":1}
I20260811 06:23:51.913455 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:51.851968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.7.4.129:41855)
0811 06:23:51.885436 (+ 33468us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:51.885457 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:51.888354 (+ 2897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:51.889073 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:51.889084 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:51.890138 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:51.891337 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.891363 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.892888 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.892893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.895422 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:51.895436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:51.896811 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:51.896819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:51.897387 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:51.898524 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:51.898549 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:51.903263 (+ 4714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:51.907560 (+ 4297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.907564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.907567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.908187 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.908675 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:51.908679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:51.908683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:51.908859 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:51.909619 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:51.909624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:51.909776 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 06:23:51.912757 (+ 2981us) client_negotiation.cc:240] Negotiation successful
0811 06:23:51.913019 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32008,"mutex_wait_us":963,"spinlock_wait_cycles":27520,"thread_start_us":31550,"threads_started":1}
I20260811 06:23:51.913829 15058 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:51.913864 15059 heartbeater.cc:347] Connected to a master server at 127.7.4.189:44951
I20260811 06:23:51.914167 15058 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:51.914216 15059 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:51.914644 15057 heartbeater.cc:347] Connected to a master server at 127.7.4.188:39139
I20260811 06:23:51.914789 15057 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:51.915454 15059 heartbeater.cc:511] Master 127.7.4.189:44951 requested a full tablet report, sending...
I20260811 06:23:51.915515 15058 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:51.915480 15057 heartbeater.cc:511] Master 127.7.4.188:39139 requested a full tablet report, sending...
I20260811 06:23:51.917770 13993 ts_manager.cc:194] Re-registered known tserver with Master: 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319)
I20260811 06:23:51.918583 14847 ts_manager.cc:194] Registered new tserver with Master: 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319)
I20260811 06:23:51.918962 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:35719
I20260811 06:23:51.919963 14644 ts_manager.cc:194] Registered new tserver with Master: 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319)
I20260811 06:23:52.006229 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14207
I20260811 06:23:52.029633 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:36963
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:35353
--webserver_port=35691
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:52.363387 15072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:52.363651 15072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:52.363827 15072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:52.370879 15072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:52.370999 15072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:52.371177 15072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:23:52.380865 15072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:36963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=35691
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:52.382181 15072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:52.383991 15072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:52.397933 15081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:52.397909 15077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:52.397986 15078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:52.398867 15072 server_base.cc:1034] running on GCE node
I20260811 06:23:52.399606 15072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:52.400736 15072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:52.401978 15072 hybrid_clock.cc:648] HybridClock initialized: now 1786429432401911 us; error 53 us; skew 500 ppm
I20260811 06:23:52.402362 15072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:52 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:52.406201 15072 init.cc:377] Logged in from keytab as oryx/127.7.4.130@KRBTEST.COM (short username oryx)
I20260811 06:23:52.407910 15072 webserver.cc:460] Webserver started at http://127.7.4.130:35691/ using document root <none> and password file <none>
I20260811 06:23:52.408776 15072 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:52.408941 15072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:52.415220 15072 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 06:23:52.417850 15088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:52.419129 15072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:23:52.419323 15072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "64bdf66c5d574b17a662db2e895a72b7"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "ebd6864db99fe8dcfd5e1f422f928c88"
server_key_iv: "f8aa8660544ca7554ae25fbdb7f49af4"
server_key_version: "encryptionkey@0"
I20260811 06:23:52.420269 15072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:52.439478 15072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:52.442935 15072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:52.443187 15072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:52.444082 15072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:52.445379 15072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:52.445497 15072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:52.445631 15072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:52.445717 15072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:52.479513 15072 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:36963
I20260811 06:23:52.479594 15207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:36963 every 8 connection(s)
I20260811 06:23:52.480921 15072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:23:52.485790 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15072
I20260811 06:23:52.486157 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14352
Aug 11 06:23:52 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
Aug 11 06:23:52 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
Aug 11 06:23:52 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429432, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
Aug 11 06:23:52 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.131@KRBTEST.COM
I20260811 06:23:52.527503 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.485149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:46845 (local address 127.7.4.189:44951)
0811 06:23:52.503912 (+ 18763us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:52.503920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:52.503941 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:23:52.504010 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:52.504016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:52.504095 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:52.504237 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:52.507354 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508630 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.511717 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.512436 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512544 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.518904 (+ 6360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:52.518912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:52.518918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:52.518964 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:52.523429 (+ 4465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524510 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.524516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.524519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.524728 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.525264 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.525270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.525273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.525730 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:52.525874 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:52.526839 (+ 965us) server_negotiation.cc:299] Negotiation successful
0811 06:23:52.527081 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18619,"thread_start_us":18492,"threads_started":1}
I20260811 06:23:52.527665 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.483949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:44945 (local address 127.7.4.190:35353)
0811 06:23:52.496489 (+ 12540us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:52.496498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:52.498663 (+ 2165us) server_negotiation.cc:407] Connection header received
0811 06:23:52.504463 (+ 5800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:52.504468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:52.504546 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:52.504630 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:52.507353 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508595 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.511728 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.512349 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512486 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.518121 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:52.518127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:52.518133 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:52.518173 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:52.523242 (+ 5069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524859 (+ 1617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.524864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.524867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.525031 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.526101 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.526106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.526109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.526456 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:52.526615 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:52.527056 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 06:23:52.527307 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12365,"thread_start_us":12541,"threads_started":1}
I20260811 06:23:52.528666 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.487316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.7.4.131:46579)
0811 06:23:52.504274 (+ 16958us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:52.504283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:52.504309 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:23:52.504376 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:52.504385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:52.504611 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:52.507146 (+ 2535us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:52.507498 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.509096 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.511014 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.511909 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512929 (+ 1020us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.519942 (+ 7013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:52.519972 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:52.519990 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:52.520043 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:52.524106 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524990 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.524998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.525002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.525186 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.525702 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.525707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.525722 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.526105 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:52.526409 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:52.527450 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0811 06:23:52.528258 (+ 808us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16757,"thread_start_us":16605,"threads_started":1}
I20260811 06:23:52.528692 15213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.484075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:45757 (local address 127.7.4.188:39139)
0811 06:23:52.496980 (+ 12905us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:52.496988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:52.501992 (+ 5004us) server_negotiation.cc:407] Connection header received
0811 06:23:52.503720 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:52.503729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:52.503845 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:52.503965 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:52.507334 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508602 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.511720 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.512466 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512553 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.519034 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:52.519042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:52.519048 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:52.519100 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:52.523249 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524422 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.524430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.524433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.524587 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:52.525511 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:52.525516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:52.525518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:52.525905 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:52.526032 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:52.528097 (+ 2065us) server_negotiation.cc:299] Negotiation successful
0811 06:23:52.528329 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12738,"thread_start_us":12628,"threads_started":1}
I20260811 06:23:52.529137 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.484703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.7.4.130:46845)
0811 06:23:52.502527 (+ 17824us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:52.502540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:52.503176 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:52.504403 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:52.504415 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:52.505871 (+ 1456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:52.507121 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.507131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508811 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.508816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.511541 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.511556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.512914 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.513266 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.514075 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:52.514105 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:52.517931 (+ 3826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:52.523619 (+ 5688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.523622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.523625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.524387 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.524862 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.524869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.525122 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.525971 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:52.525978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:52.526384 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 06:23:52.527717 (+ 1333us) client_negotiation.cc:240] Negotiation successful
0811 06:23:52.527996 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17663,"mutex_wait_us":3239,"spinlock_wait_cycles":18176,"thread_start_us":14847,"threads_started":1}
I20260811 06:23:52.529171 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.484707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:44945)
0811 06:23:52.498455 (+ 13748us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:52.498516 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:52.503161 (+ 4645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:52.504855 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:52.504859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:52.505894 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:52.507072 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.507103 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508807 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.508813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.511562 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.511577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.512892 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.512901 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.513262 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.514003 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:52.514038 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:52.517907 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:52.524057 (+ 6150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.524064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.524683 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.525161 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.525167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.525170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.525776 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.526597 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:52.526604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:52.526784 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 06:23:52.526967 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 06:23:52.527521 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13329,"mutex_wait_us":368,"thread_start_us":13034,"threads_started":1}
I20260811 06:23:52.529763 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:52.487167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.7.4.130:45757)
0811 06:23:52.501856 (+ 14689us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:52.501885 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:52.503375 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:52.504403 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:52.504415 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:52.505871 (+ 1456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:52.507071 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.507106 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.508818 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.508822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.511541 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:52.511556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:52.514040 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:52.514048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:52.514256 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:52.515007 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:52.515038 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:52.518782 (+ 3744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:52.523455 (+ 4673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.523463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.523479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.524270 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.524735 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:52.524739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:52.524742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:52.525334 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:52.526702 (+ 1368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:52.526710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:52.526845 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 06:23:52.528935 (+ 2090us) client_negotiation.cc:240] Negotiation successful
0811 06:23:52.529241 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14530,"mutex_wait_us":426,"spinlock_wait_cycles":10624,"thread_start_us":14341,"threads_started":1}
I20260811 06:23:52.530711 15209 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:52.530952 15208 heartbeater.cc:347] Connected to a master server at 127.7.4.188:39139
I20260811 06:23:52.531075 15209 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:52.531106 15208 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:52.531389 14480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:52.532229 15209 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:52.533315 15210 heartbeater.cc:347] Connected to a master server at 127.7.4.189:44951
I20260811 06:23:52.533473 15210 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:52.534065 15210 heartbeater.cc:511] Master 127.7.4.189:44951 requested a full tablet report, sending...
I20260811 06:23:52.535398 14644 ts_manager.cc:194] Registered new tserver with Master: 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:52.535462 13993 ts_manager.cc:194] Re-registered known tserver with Master: 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:52.536950 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:44945
I20260811 06:23:52.532204 15208 heartbeater.cc:511] Master 127.7.4.188:39139 requested a full tablet report, sending...
I20260811 06:23:52.538735 14847 ts_manager.cc:194] Registered new tserver with Master: 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:52.618348 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14352
I20260811 06:23:52.639719 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:46579
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:35353
--webserver_port=40947
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:23:52.923017 15058 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
W20260811 06:23:52.992131 15223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:52.992463 15223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:52.992640 15223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:23:53.000454 15223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:23:53.000550 15223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:23:53.000738 15223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:23:53.010747 15223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:46579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=40947
--tserver_master_addrs=127.7.4.190:35353,127.7.4.189:44951,127.7.4.188:39139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:23:53.012125 15223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:23:53.013509 15223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:23:53.032696 15230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:53.033044 15229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:23:53.033571 15233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:23:53.034024 15223 server_base.cc:1034] running on GCE node
I20260811 06:23:53.034933 15223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:23:53.036167 15223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:23:53.037390 15223 hybrid_clock.cc:648] HybridClock initialized: now 1786429433037298 us; error 83 us; skew 500 ppm
I20260811 06:23:53.037747 15223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:23:53 dist-test-slave-07c2 krb5kdc[13942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:23:53.041142 15223 init.cc:377] Logged in from keytab as oryx/127.7.4.131@KRBTEST.COM (short username oryx)
I20260811 06:23:53.044708 15223 webserver.cc:460] Webserver started at http://127.7.4.131:40947/ using document root <none> and password file <none>
I20260811 06:23:53.045379 15223 fs_manager.cc:359] Metadata directory not provided
I20260811 06:23:53.045560 15223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:23:53.053894 15223 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:23:53.057250 15241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:53.058738 15223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:23:53.058938 15223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "b84676b31ab348beb6591e1c6c605573"
format_stamp: "Formatted at 2026-08-11 06:23:49 on dist-test-slave-07c2"
server_key: "6365a0876ffd9b8bf4b4e8a18d7f0350"
server_key_iv: "d3859b7842dcccc12a73f23f022b25bc"
server_key_version: "encryptionkey@0"
I20260811 06:23:53.059993 15223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:23:53.085175 15223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:23:53.092723 15223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:23:53.093048 15223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:23:53.094463 15223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:23:53.096426 15223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:23:53.096546 15223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:53.096661 15223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:23:53.096748 15223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:23:53.145543 15223 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46579
I20260811 06:23:53.145629 15361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46579 every 8 connection(s)
I20260811 06:23:53.147706 15223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:23:53.157492 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15223
Aug 11 06:23:53 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.189@KRBTEST.COM
Aug 11 06:23:53 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.188@KRBTEST.COM
I20260811 06:23:53.189232 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.152078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.7.4.131:60511)
0811 06:23:53.153244 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.153305 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.157355 (+ 4050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.158282 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.158292 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.160374 (+ 2082us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:53.160682 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.160692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.162264 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.162271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.164606 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.164624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.170199 (+ 5575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.170206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.170407 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.171103 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:53.171124 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:53.175981 (+ 4857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:53.181086 (+ 5105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.181097 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.181113 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.181894 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.182465 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.182469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.182473 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.182896 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.183989 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:53.183997 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:53.184558 (+ 561us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.185018 (+ 460us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.185838 (+ 820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":4933,"threads_started":1}
I20260811 06:23:53.191965 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.154074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:60511 (local address 127.7.4.188:39139)
0811 06:23:53.156802 (+ 2728us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.156808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.156828 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:23:53.157780 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.157785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.157879 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.159087 (+ 1208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.160835 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.162116 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.168870 (+ 6754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.169787 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.169923 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.176222 (+ 6299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.176231 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.176238 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.176286 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.180935 (+ 4649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.182064 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.182081 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.182087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.182309 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.183107 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.183116 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.183123 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.183671 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.183873 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.191136 (+ 7263us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.191458 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2600,"thread_start_us":3221,"threads_started":1}
I20260811 06:23:53.193665 15362 heartbeater.cc:347] Connected to a master server at 127.7.4.188:39139
I20260811 06:23:53.194047 15362 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:53.195274 15362 heartbeater.cc:511] Master 127.7.4.188:39139 requested a full tablet report, sending...
I20260811 06:23:53.195389 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.153241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:50117 (local address 127.7.4.189:44951)
0811 06:23:53.158132 (+ 4891us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.158138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.158155 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:23:53.158213 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.158217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.158293 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.160913 (+ 2620us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.161127 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.164895 (+ 3768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.167158 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.168021 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.168175 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.175244 (+ 7069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.175257 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.175266 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.176191 (+ 925us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.186332 (+ 10141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.189981 (+ 3649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.189987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.189991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.190248 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.192229 (+ 1981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.192234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.192237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.192684 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.192848 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.194571 (+ 1723us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.194855 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4753,"thread_start_us":2494,"threads_started":1}
I20260811 06:23:53.196869 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.153134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.7.4.131:50117)
0811 06:23:53.153994 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.154012 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.157772 (+ 3760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.158462 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.158466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.159125 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:53.160117 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.160148 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.165075 (+ 4927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.165083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.166942 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.166960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.168558 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.168569 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.168908 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.169835 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:53.169872 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:53.174797 (+ 4925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:53.186516 (+ 11719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.186524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.186531 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.187448 (+ 917us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.190455 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.190462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.190469 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.190732 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.192929 (+ 2197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:53.192934 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:53.193119 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.194135 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.196406 (+ 2271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":2898,"threads_started":1}
Aug 11 06:23:53 dist-test-slave-07c2 krb5kdc[13942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429433, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:23:53.197569 14847 ts_manager.cc:194] Registered new tserver with Master: b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579)
I20260811 06:23:53.198077 15365 heartbeater.cc:347] Connected to a master server at 127.7.4.189:44951
I20260811 06:23:53.198210 15365 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:53.198968 15365 heartbeater.cc:511] Master 127.7.4.189:44951 requested a full tablet report, sending...
I20260811 06:23:53.200544 14644 ts_manager.cc:194] Registered new tserver with Master: b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579)
I20260811 06:23:53.209389 15372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.154793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:60797)
0811 06:23:53.155788 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.155812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.176087 (+ 20275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.178169 (+ 2082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.178176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.178828 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:23:53.179449 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.179467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.186380 (+ 6913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.186388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.191532 (+ 5144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.191552 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.193266 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.193274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.193450 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.194250 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:23:53.194272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:23:53.197986 (+ 3714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:23:53.205226 (+ 7240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.205233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.205240 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.206015 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.206554 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:23:53.206561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:23:53.206564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:23:53.206828 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:23:53.207948 (+ 1120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:23:53.207955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:23:53.208138 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.208298 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.208864 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":2052,"threads_started":1}
I20260811 06:23:53.210687 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.153733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:60797 (local address 127.7.4.190:35353)
0811 06:23:53.156282 (+ 2549us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.156288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.176314 (+ 20026us) server_negotiation.cc:407] Connection header received
0811 06:23:53.176623 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.176631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.176740 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.176884 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.181575 (+ 4691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.183236 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.191947 (+ 8711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.192667 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.192795 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.199550 (+ 6755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.199558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.199563 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.199611 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.205084 (+ 5473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.206235 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.206243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.206249 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.206410 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.207263 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.207270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.207274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.207705 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.207870 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.209938 (+ 2068us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.210199 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2360,"thread_start_us":2252,"threads_started":1}
I20260811 06:23:53.211570 15363 heartbeater.cc:347] Connected to a master server at 127.7.4.190:35353
I20260811 06:23:53.211735 15363 heartbeater.cc:464] Registering TS with master...
I20260811 06:23:53.212244 15363 heartbeater.cc:511] Master 127.7.4.190:35353 requested a full tablet report, sending...
I20260811 06:23:53.213433 13993 ts_manager.cc:194] Re-registered known tserver with Master: b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579)
I20260811 06:23:53.214591 13993 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:60797
I20260811 06:23:53.223485 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.163333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.7.4.188:39139)
0811 06:23:53.190225 (+ 26892us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.190231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.192304 (+ 2073us) server_negotiation.cc:407] Connection header received
0811 06:23:53.195872 (+ 3568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.195881 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.195983 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.196115 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.200140 (+ 4025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.201511 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.206029 (+ 4518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.206802 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.206936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.214608 (+ 7672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.214613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.214616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.214646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.220115 (+ 5469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.220769 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.220773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.220775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.221097 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.221478 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.221483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.221486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.221881 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.222157 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.222707 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.222989 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26711,"thread_start_us":26564,"threads_started":1}
I20260811 06:23:53.225574 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.163101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.7.4.189:44951)
0811 06:23:53.192840 (+ 29739us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.192846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.192865 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:23:53.192923 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.192928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.193000 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.193104 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.196736 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.201288 (+ 4552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.210538 (+ 9250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.211705 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.211823 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.216792 (+ 4969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.216800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.216805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.216844 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.221493 (+ 4649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.222320 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.222328 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.222334 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.222500 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.223029 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.223034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.223037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.223417 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.224808 (+ 1391us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.224935 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.225159 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29598,"thread_start_us":31052,"threads_started":1}
I20260811 06:23:53.228070 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.162787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45962 (local address 127.7.4.190:35353)
0811 06:23:53.174300 (+ 11513us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.174305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.204220 (+ 29915us) server_negotiation.cc:407] Connection header received
0811 06:23:53.204490 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.204496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.204588 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.204673 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.210481 (+ 5808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.211961 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.215034 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.215839 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.215986 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.219716 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.219724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.219728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.219764 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.224411 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.225688 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.225693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.225696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.225869 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.226295 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.226301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.226304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.226709 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.226935 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.227320 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.227584 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11371,"thread_start_us":9034,"threads_started":1}
I20260811 06:23:53.232178 13993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:53.239845 13993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:53.271657 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.258818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:37319 (local address 127.0.0.1:48198)
0811 06:23:53.260173 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.260196 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.260357 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.261906 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.261914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.261949 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:53.262479 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.262495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.265534 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.265543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.268643 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.268662 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.270708 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.270714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.270883 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.270919 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.270994 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.271188 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":10994,"threads_started":1}
I20260811 06:23:53.273202 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.259049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.7.4.129:37319)
0811 06:23:53.261115 (+ 2066us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.261138 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.261175 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:23:53.261304 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.261316 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.261644 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.262727 (+ 1083us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:53.263016 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.264804 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.268847 (+ 4043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.270462 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.272417 (+ 1955us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.272637 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.272706 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.272875 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1835,"thread_start_us":1684,"threads_started":1}
I20260811 06:23:53.277254 14985 tablet_service.cc:1467] Processing CreateTablet for tablet 356353be62c646708455fa8d91839cfd (DEFAULT_TABLE table=test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:53.278499 14985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356353be62c646708455fa8d91839cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:53.281210 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.259242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36963 (local address 127.0.0.1:52478)
0811 06:23:53.270140 (+ 10898us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.270165 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.270451 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.271452 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.271457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.271481 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:53.271963 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.271977 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.274553 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.274560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.276970 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.277011 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.280133 (+ 3122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.280142 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.280399 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.280452 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.280539 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.280731 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10740,"thread_start_us":1770,"threads_started":1}
I20260811 06:23:53.282562 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.260509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.7.4.130:36963)
0811 06:23:53.265395 (+ 4886us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.265407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.270413 (+ 5006us) server_negotiation.cc:407] Connection header received
0811 06:23:53.270656 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.270667 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.270908 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.271205 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:53.272170 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.274372 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.277153 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.279587 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.281444 (+ 1857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.281762 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.281856 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.282072 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4688,"thread_start_us":4580,"threads_started":1}
I20260811 06:23:53.286298 15139 tablet_service.cc:1467] Processing CreateTablet for tablet 356353be62c646708455fa8d91839cfd (DEFAULT_TABLE table=test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:53.287824 15139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356353be62c646708455fa8d91839cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:53.293665 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.260193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46579 (local address 127.0.0.1:40216)
0811 06:23:53.271960 (+ 11767us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.271978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.272080 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.288686 (+ 16606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.288692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.288729 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:53.288901 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.288912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.290203 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.290210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.291496 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.291517 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.292827 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.292833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.293002 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.293051 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.293173 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.293334 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11649,"thread_start_us":15415,"threads_started":1}
I20260811 06:23:53.293783 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.260442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.7.4.131:46579)
0811 06:23:53.288153 (+ 27711us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.288159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.288186 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:23:53.288265 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.288270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.288427 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.288698 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:53.289037 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.290071 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.291662 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.292659 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.292809 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.293043 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.293267 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.293426 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27523,"thread_start_us":23036,"threads_started":1}
I20260811 06:23:53.296257 15400 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Bootstrap starting.
I20260811 06:23:53.297497 15294 tablet_service.cc:1467] Processing CreateTablet for tablet 356353be62c646708455fa8d91839cfd (DEFAULT_TABLE table=test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:53.299058 15294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356353be62c646708455fa8d91839cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:53.300505 15400 tablet_bootstrap.cc:654] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:53.302192 15400 log.cc:824] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:53.303210 15401 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Bootstrap starting.
I20260811 06:23:53.306406 15401 tablet_bootstrap.cc:654] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:53.306950 15400 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: No bootstrap required, opened a new log
I20260811 06:23:53.307235 15400 ts_tablet_manager.cc:1397] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 06:23:53.307588 15401 log.cc:824] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:53.311168 15401 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: No bootstrap required, opened a new log
I20260811 06:23:53.311499 15401 ts_tablet_manager.cc:1397] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 06:23:53.316090 15404 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Bootstrap starting.
I20260811 06:23:53.318056 15400 raft_consensus.cc:348] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.318427 15400 raft_consensus.cc:374] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:53.318557 15400 raft_consensus.cc:729] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124c93e3f1e9482eaf775e723d857159, State: Initialized, Role: FOLLOWER
I20260811 06:23:53.319167 15400 consensus_queue.cc:260] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.319887 15404 tablet_bootstrap.cc:654] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:53.320892 15400 ts_tablet_manager.cc:1428] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 06:23:53.321563 15404 log.cc:824] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Log is configured to *not* fsync() on all Append() calls
I20260811 06:23:53.324028 15401 raft_consensus.cc:348] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.324525 15401 raft_consensus.cc:374] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:53.324637 15401 raft_consensus.cc:729] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bdf66c5d574b17a662db2e895a72b7, State: Initialized, Role: FOLLOWER
I20260811 06:23:53.325217 15401 consensus_queue.cc:260] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.326949 15209 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
I20260811 06:23:53.327565 15401 ts_tablet_manager.cc:1428] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Time spent starting tablet: real 0.016s user 0.007s sys 0.006s
I20260811 06:23:53.332549 15404 tablet_bootstrap.cc:492] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: No bootstrap required, opened a new log
I20260811 06:23:53.332818 15404 ts_tablet_manager.cc:1397] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s
I20260811 06:23:53.341051 15404 raft_consensus.cc:348] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.341414 15404 raft_consensus.cc:374] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:53.341506 15404 raft_consensus.cc:729] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84676b31ab348beb6591e1c6c605573, State: Initialized, Role: FOLLOWER
I20260811 06:23:53.342027 15404 consensus_queue.cc:260] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.343178 15363 heartbeater.cc:503] Master 127.7.4.190:35353 was elected leader, sending a full tablet report...
I20260811 06:23:53.343801 15404 ts_tablet_manager.cc:1428] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
W20260811 06:23:53.383929 15060 tablet.cc:2369] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:53.402269 15366 tablet.cc:2369] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:23:53.490844 15211 tablet.cc:2369] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:23:53.542810 15408 raft_consensus.cc:482] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:53.543176 15408 raft_consensus.cc:504] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.545050 15408 leader_election.cc:288] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319), 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:53.551227 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.545441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36963 (local address 127.7.4.131:39655)
0811 06:23:53.545858 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.545875 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.546008 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.546344 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.546348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.546372 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:53.546589 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.546598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.547673 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.547678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.549163 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.549175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.550456 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.550463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.550662 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.550686 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.550793 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.550937 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":119}
I20260811 06:23:53.551249 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.545569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39655 (local address 127.7.4.130:36963)
0811 06:23:53.545821 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.545829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.545962 (+ 133us) server_negotiation.cc:407] Connection header received
0811 06:23:53.546159 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.546163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.546226 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.546327 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:53.546739 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.547531 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.549336 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.550277 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.550433 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.550543 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.550821 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.550950 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 06:23:53.552168 15372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.545441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:37319 (local address 127.7.4.131:45189)
0811 06:23:53.545640 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:53.545682 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:53.545845 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:53.546429 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:53.546434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:53.546464 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:53.546731 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.546741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.547988 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.547996 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.549435 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:53.549448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.551269 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.551275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:53.551640 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.551675 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:23:53.551804 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 06:23:53.551956 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":28}
I20260811 06:23:53.552290 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.545781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:45189 (local address 127.7.4.129:37319)
0811 06:23:53.546091 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.546096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.546118 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:23:53.546183 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.546187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.546266 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.546411 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:53.546889 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.547825 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.549608 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.550893 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.551030 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.551127 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.551835 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.552016 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:23:53.552978 15160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356353be62c646708455fa8d91839cfd" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdf66c5d574b17a662db2e895a72b7" is_pre_election: true
I20260811 06:23:53.553406 15009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356353be62c646708455fa8d91839cfd" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124c93e3f1e9482eaf775e723d857159" is_pre_election: true
I20260811 06:23:53.553547 15160 raft_consensus.cc:2454] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 0.
I20260811 06:23:53.553946 15009 raft_consensus.cc:2454] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 0.
I20260811 06:23:53.554332 15243 leader_election.cc:302] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64bdf66c5d574b17a662db2e895a72b7, b84676b31ab348beb6591e1c6c605573; no voters:
I20260811 06:23:53.554895 15408 raft_consensus.cc:2781] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:53.555078 15408 raft_consensus.cc:482] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:53.555181 15408 raft_consensus.cc:3037] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:53.557688 15408 raft_consensus.cc:504] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.558539 15408 leader_election.cc:288] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 election: Requested vote from peers 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319), 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:53.558956 15009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356353be62c646708455fa8d91839cfd" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124c93e3f1e9482eaf775e723d857159"
I20260811 06:23:53.559055 15160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356353be62c646708455fa8d91839cfd" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdf66c5d574b17a662db2e895a72b7"
I20260811 06:23:53.559159 15009 raft_consensus.cc:3037] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:53.559316 15160 raft_consensus.cc:3037] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:53.561522 15009 raft_consensus.cc:2454] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 1.
I20260811 06:23:53.561673 15160 raft_consensus.cc:2454] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 1.
I20260811 06:23:53.561926 15244 leader_election.cc:302] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 124c93e3f1e9482eaf775e723d857159, b84676b31ab348beb6591e1c6c605573; no voters:
I20260811 06:23:53.562224 15408 raft_consensus.cc:2781] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:53.562633 15408 raft_consensus.cc:686] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 1 LEADER]: Becoming Leader. State: Replica: b84676b31ab348beb6591e1c6c605573, State: Running, Role: LEADER
I20260811 06:23:53.563365 15408 consensus_queue.cc:237] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } }
I20260811 06:23:53.568621 13993 catalog_manager.cc:5495] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 reported cstate change: term changed from 0 to 1, leader changed from <none> to b84676b31ab348beb6591e1c6c605573 (127.7.4.131). New cstate: current_term: 1 leader_uuid: "b84676b31ab348beb6591e1c6c605573" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } health_report { overall_health: HEALTHY } } }
I20260811 06:23:53.634307 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.629201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.7.4.131:46579)
0811 06:23:53.629386 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.629390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.629408 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:23:53.629577 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.629581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.629656 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.629789 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:53.630141 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.630876 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.632284 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.632785 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.632874 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.633283 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:53.633749 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:53.633856 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.634026 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.634089 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:23:53.642340 15009 raft_consensus.cc:1261] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Refusing update from remote peer b84676b31ab348beb6591e1c6c605573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:53.642362 15160 raft_consensus.cc:1261] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Refusing update from remote peer b84676b31ab348beb6591e1c6c605573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:53.643378 15409 consensus_queue.cc:1035] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:53.643728 15408 consensus_queue.cc:1035] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:53.658289 15413 mvcc.cc:205] Tried to move back new op lower bound from 7317214960189145088 to 7317214959879528448. Current Snapshot: MvccSnapshot[applied={T|T < 7317214960189145088}]
I20260811 06:23:53.678143 13993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:53.678464 13993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:53.685122 13993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a86a659668f8494db15ce16bc2d7a1c4: Sending DeleteTablet for 3 replicas of tablet 356353be62c646708455fa8d91839cfd
I20260811 06:23:53.686225 14985 tablet_service.cc:1514] Processing DeleteTablet for tablet 356353be62c646708455fa8d91839cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:53 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:48198
I20260811 06:23:53.686805 15139 tablet_service.cc:1514] Processing DeleteTablet for tablet 356353be62c646708455fa8d91839cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:53 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:52478
I20260811 06:23:53.687958 15424 tablet_replica.cc:330] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: stopping tablet replica
I20260811 06:23:53.688282 15423 tablet_replica.cc:330] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: stopping tablet replica
I20260811 06:23:53.688716 15294 tablet_service.cc:1514] Processing DeleteTablet for tablet 356353be62c646708455fa8d91839cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:53 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:40216
I20260811 06:23:53.689872 15423 raft_consensus.cc:2229] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:53.690419 15423 raft_consensus.cc:2258] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:53.692929 15423 ts_tablet_manager.cc:1905] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:53.694897 15424 raft_consensus.cc:2229] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:53.695472 15424 raft_consensus.cc:2258] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:53.699059 15426 tablet_replica.cc:330] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: stopping tablet replica
I20260811 06:23:53.699733 15426 raft_consensus.cc:2229] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:53.701574 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 13959
I20260811 06:23:53.702559 14027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:53.702654 15424 ts_tablet_manager.cc:1905] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:53.704887 15426 raft_consensus.cc:2258] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:53.706265 15423 ts_tablet_manager.cc:1918] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:53.706425 15423 log.cc:1196] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/356353be62c646708455fa8d91839cfd
I20260811 06:23:53.707034 15423 ts_tablet_manager.cc:1939] T 356353be62c646708455fa8d91839cfd P 124c93e3f1e9482eaf775e723d857159: Deleting consensus metadata
I20260811 06:23:53.707399 15426 ts_tablet_manager.cc:1905] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:53.715075 15424 ts_tablet_manager.cc:1918] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:53.715363 15424 log.cc:1196] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/356353be62c646708455fa8d91839cfd
I20260811 06:23:53.716005 15424 ts_tablet_manager.cc:1939] T 356353be62c646708455fa8d91839cfd P 64bdf66c5d574b17a662db2e895a72b7: Deleting consensus metadata
I20260811 06:23:53.722983 15426 ts_tablet_manager.cc:1918] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:53.723233 15426 log.cc:1196] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/356353be62c646708455fa8d91839cfd
I20260811 06:23:53.724025 15426 ts_tablet_manager.cc:1939] T 356353be62c646708455fa8d91839cfd P b84676b31ab348beb6591e1c6c605573: Deleting consensus metadata
I20260811 06:23:53.728797 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.689411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.7.4.189:44951)
0811 06:23:53.689611 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.689617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.700531 (+ 10914us) server_negotiation.cc:407] Connection header received
0811 06:23:53.701055 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.701061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.701144 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.701249 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.704671 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.705751 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.713745 (+ 7994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.714395 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.714531 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.721334 (+ 6803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.721341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.721346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.721391 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.725747 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.726573 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.726579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.726582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.726732 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.727108 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.727113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.727116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.727490 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.727846 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.728190 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.728446 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:23:53.743448 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.700568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.7.4.188:39139)
0811 06:23:53.700757 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.700763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.728983 (+ 28220us) server_negotiation.cc:407] Connection header received
0811 06:23:53.729075 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.729079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.729148 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.729262 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.731039 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.732134 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.733669 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.734196 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.734338 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.737762 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.737767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.737770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.737807 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.740644 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.741307 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.741314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.741318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.741450 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.741953 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.741959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.741962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.742244 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.742503 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.742808 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.743054 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:23:53.871937 13976 catalog_manager.cc:4841] TS b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579): tablet 356353be62c646708455fa8d91839cfd (table test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]) successfully deleted
I20260811 06:23:53.872404 13977 catalog_manager.cc:4841] TS 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963): tablet 356353be62c646708455fa8d91839cfd (table test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]) successfully deleted
I20260811 06:23:53.873473 13978 catalog_manager.cc:4841] TS 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319): tablet 356353be62c646708455fa8d91839cfd (table test-table [id=a57b5c84b0424ff6b3428d1e3b1a4d53]) successfully deleted
W20260811 06:23:53.873674 13978 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.4.129:37319, user_credentials={real_user=slave}} blocked reactor thread for 164748us
I20260811 06:23:53.874516 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 13959
I20260811 06:23:53.939688 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.907148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.7.4.188:39139)
0811 06:23:53.907612 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.907622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.917950 (+ 10328us) server_negotiation.cc:407] Connection header received
0811 06:23:53.918603 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.918613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.918751 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.918845 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.921995 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.923516 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.927132 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.927830 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.927938 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.931639 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.931647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.931652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.931695 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.936012 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.936983 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.936989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.936993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.937163 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.937652 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.937658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.937662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.938068 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.938311 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.939000 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.939264 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260811 06:23:53.943432 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:53.907752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.7.4.189:44951)
0811 06:23:53.907998 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:53.908006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:53.926930 (+ 18924us) server_negotiation.cc:407] Connection header received
0811 06:23:53.927578 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:53.927585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:53.927693 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:53.927811 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:53.930092 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.930837 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.932628 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:53.933339 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:53.933498 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:53.936717 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:53.936727 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:53.936731 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:53.936765 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:53.940507 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.941238 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.941244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.941247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.941386 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:53.941884 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:53.941889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:53.941894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:53.942187 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:53.942453 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:53.942809 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 06:23:53.943046 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:23:54.666424 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:54.665397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:46749)
0811 06:23:54.666054 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:54.666166 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":467,"threads_started":1}
W20260811 06:23:54.666913 15363 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:35353 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:54.673250 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:54.672524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:51203)
0811 06:23:54.673001 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:54.673102 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":272,"threads_started":1}
W20260811 06:23:54.673734 15058 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:35353 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:54.673830 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:54.673125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:50885)
0811 06:23:54.673582 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:54.673685 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":251,"threads_started":1}
W20260811 06:23:54.674264 15209 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:35353 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.185940 15440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a86a659668f8494db15ce16bc2d7a1c4)
I20260811 06:23:55.186304 15440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:55.186466 15442 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a86a659668f8494db15ce16bc2d7a1c4)
I20260811 06:23:55.186744 15442 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:55.187996 15440 leader_election.cc:288] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), fd0fca5968034acd953d7416d6009419 (127.7.4.189:44951)
I20260811 06:23:55.188127 15442 leader_election.cc:288] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), 68891d8213204e0591f67e622bfc2aff (127.7.4.188:39139)
I20260811 06:23:55.200716 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.188071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46398)
0811 06:23:55.200321 (+ 12250us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.200479 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12116,"threads_started":1}
I20260811 06:23:55.200695 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.188606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46402)
0811 06:23:55.200216 (+ 11610us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.200409 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11476,"thread_start_us":11332,"threads_started":1}
W20260811 06:23:55.200959 14628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
W20260811 06:23:55.203572 14829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.204131 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.202521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46406)
0811 06:23:55.203751 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.203895 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1118}
W20260811 06:23:55.204303 14628 leader_election.cc:334] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.206473 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.205781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46416)
0811 06:23:55.205974 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.206266 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:23:55.206720 14829 leader_election.cc:334] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.210533 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.188585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.7.4.189:44951)
0811 06:23:55.203314 (+ 14729us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:55.203319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:55.203340 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:23:55.203411 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:55.203418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:55.203508 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:55.204175 (+ 667us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:55.204459 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.205769 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.208178 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.209541 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.209688 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:55.209781 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:55.210197 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 06:23:55.210300 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":14540,"thread_start_us":14363,"threads_started":1}
I20260811 06:23:55.210664 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.188264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44951 (local address 127.0.0.1:43992)
0811 06:23:55.201187 (+ 12923us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.201207 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:55.201374 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:55.203705 (+ 2331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:55.203710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:55.203741 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:55.204021 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:55.204035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.205946 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.205961 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:55.208022 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:55.208033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.209737 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.209748 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:55.209965 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:55.210062 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 06:23:55.210163 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:23:55.210308 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12822,"mutex_wait_us":88,"thread_start_us":12621,"threads_started":1}
I20260811 06:23:55.211488 14654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68891d8213204e0591f67e622bfc2aff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd0fca5968034acd953d7416d6009419" is_pre_election: true
I20260811 06:23:55.211867 14654 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68891d8213204e0591f67e622bfc2aff in term 1.
I20260811 06:23:55.212069 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.191418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.7.4.188:39139)
0811 06:23:55.205033 (+ 13615us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:55.205039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:55.205056 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:23:55.205116 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:55.205120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:55.205192 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:55.205305 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:55.205877 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.207500 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.209699 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.211041 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.211180 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:55.211315 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:55.211603 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 06:23:55.211728 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":13497,"thread_start_us":13339,"threads_started":1}
I20260811 06:23:55.212105 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.197934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:39139 (local address 127.0.0.1:34712)
0811 06:23:55.202516 (+ 4582us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.202533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:55.202655 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:55.205333 (+ 2678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:55.205338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:55.205387 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:55.205710 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:55.205724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.207636 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.207640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:55.209541 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:55.209553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:55.211228 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:55.211234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:55.211443 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:55.211469 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:23:55.211577 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:23:55.211726 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4480,"mutex_wait_us":54,"thread_start_us":5555,"threads_started":1}
I20260811 06:23:55.212641 14830 leader_election.cc:302] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68891d8213204e0591f67e622bfc2aff, fd0fca5968034acd953d7416d6009419; no voters: a86a659668f8494db15ce16bc2d7a1c4
I20260811 06:23:55.212736 14858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd0fca5968034acd953d7416d6009419" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68891d8213204e0591f67e622bfc2aff" is_pre_election: true
I20260811 06:23:55.213040 14858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd0fca5968034acd953d7416d6009419 in term 1.
I20260811 06:23:55.213306 15440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:23:55.213441 15440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Starting leader election (detected failure of leader a86a659668f8494db15ce16bc2d7a1c4)
I20260811 06:23:55.213551 15440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:23:55.213533 14627 leader_election.cc:302] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68891d8213204e0591f67e622bfc2aff, fd0fca5968034acd953d7416d6009419; no voters: a86a659668f8494db15ce16bc2d7a1c4
I20260811 06:23:55.213922 15442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:23:55.214063 15442 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a86a659668f8494db15ce16bc2d7a1c4)
I20260811 06:23:55.214184 15442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:23:55.216140 15440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:55.216898 15440 leader_election.cc:288] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 election: Requested vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), fd0fca5968034acd953d7416d6009419 (127.7.4.189:44951)
I20260811 06:23:55.216840 15442 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:55.217523 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.217129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46430)
0811 06:23:55.217277 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.217357 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 06:23:55.217494 14654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68891d8213204e0591f67e622bfc2aff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd0fca5968034acd953d7416d6009419"
I20260811 06:23:55.217877 14654 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 68891d8213204e0591f67e622bfc2aff in current term 2: Already voted for candidate fd0fca5968034acd953d7416d6009419 in this term.
I20260811 06:23:55.218202 15442 leader_election.cc:288] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 election: Requested vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), 68891d8213204e0591f67e622bfc2aff (127.7.4.188:39139)
I20260811 06:23:55.218396 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.217653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46438)
0811 06:23:55.217843 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.217944 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 06:23:55.218654 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.218280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46452)
0811 06:23:55.218439 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.218532 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 06:23:55.218930 14829 leader_election.cc:334] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.219098 14829 leader_election.cc:302] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68891d8213204e0591f67e622bfc2aff; no voters: a86a659668f8494db15ce16bc2d7a1c4, fd0fca5968034acd953d7416d6009419
I20260811 06:23:55.219409 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.218960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46456)
0811 06:23:55.219158 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.219249 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 06:23:55.219347 14858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd0fca5968034acd953d7416d6009419" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68891d8213204e0591f67e622bfc2aff"
W20260811 06:23:55.219558 14628 leader_election.cc:334] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:55.219658 14858 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fd0fca5968034acd953d7416d6009419 in current term 2: Already voted for candidate 68891d8213204e0591f67e622bfc2aff in this term.
I20260811 06:23:55.219986 15440 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 06:23:55.220091 14627 leader_election.cc:302] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd0fca5968034acd953d7416d6009419; no voters: 68891d8213204e0591f67e622bfc2aff, a86a659668f8494db15ce16bc2d7a1c4
I20260811 06:23:55.220357 15442 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 06:23:55.668357 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.667582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:37285)
0811 06:23:55.668111 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.668189 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":287,"threads_started":1}
I20260811 06:23:55.675069 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.674354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:45691)
0811 06:23:55.674823 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.674899 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":269,"threads_started":1}
I20260811 06:23:55.675582 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:55.674836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:55701)
0811 06:23:55.675318 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:55.675411 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":264,"threads_started":1}
I20260811 06:23:56.670176 15454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:56.669264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:60585)
0811 06:23:56.669764 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:56.669886 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260811 06:23:56.676609 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:56.675888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:33449)
0811 06:23:56.676351 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:56.676444 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":235,"threads_started":1}
I20260811 06:23:56.677260 15456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:56.676396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:52869)
0811 06:23:56.677040 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:56.677116 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":269,"threads_started":1}
I20260811 06:23:57.096827 15458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:57.097117 15458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:57.097774 15458 leader_election.cc:288] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), fd0fca5968034acd953d7416d6009419 (127.7.4.189:44951)
I20260811 06:23:57.098517 14654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68891d8213204e0591f67e622bfc2aff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd0fca5968034acd953d7416d6009419" is_pre_election: true
I20260811 06:23:57.098770 14654 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68891d8213204e0591f67e622bfc2aff in term 2.
I20260811 06:23:57.099138 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.098036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46740)
0811 06:23:57.098753 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.098848 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":430,"threads_started":1}
I20260811 06:23:57.099334 14830 leader_election.cc:302] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68891d8213204e0591f67e622bfc2aff, fd0fca5968034acd953d7416d6009419; no voters:
I20260811 06:23:57.099707 15458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 06:23:57.099855 15458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:57.099984 15458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:23:57.100666 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.100335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46742)
0811 06:23:57.100471 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.100546 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:23:57.100870 14829 leader_election.cc:334] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:57.102497 15458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:57.103098 15458 leader_election.cc:288] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 election: Requested vote from peers a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353), fd0fca5968034acd953d7416d6009419 (127.7.4.189:44951)
I20260811 06:23:57.103546 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.103135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46756)
0811 06:23:57.103348 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.103428 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 06:23:57.103621 14654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68891d8213204e0591f67e622bfc2aff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd0fca5968034acd953d7416d6009419"
I20260811 06:23:57.103865 14654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:23:57.104562 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.104285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46772)
0811 06:23:57.104405 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.104475 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:23:57.104796 14829 leader_election.cc:334] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
I20260811 06:23:57.106247 14654 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68891d8213204e0591f67e622bfc2aff in term 3.
I20260811 06:23:57.106626 14830 leader_election.cc:302] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68891d8213204e0591f67e622bfc2aff, fd0fca5968034acd953d7416d6009419; no voters: a86a659668f8494db15ce16bc2d7a1c4
I20260811 06:23:57.106894 15458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:23:57.107276 15458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [term 3 LEADER]: Becoming Leader. State: Replica: 68891d8213204e0591f67e622bfc2aff, State: Running, Role: LEADER
I20260811 06:23:57.107717 15458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } }
I20260811 06:23:57.109303 15461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68891d8213204e0591f67e622bfc2aff. Latest consensus state: current_term: 3 leader_uuid: "68891d8213204e0591f67e622bfc2aff" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:57.109612 15461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:57.110263 15463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:23:57.114033 15463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:23:57.114864 15463 catalog_manager.cc:1259] Loaded cluster ID: 2da51b12747e41389af652dcb0bad33e
I20260811 06:23:57.114966 15463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:23:57.116185 15463 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:23:57.116986 15463 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Loaded TSK: 0
I20260811 06:23:57.117872 15463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:23:57.134954 14847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:23:57.142659 14847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:23:57.146948 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.146482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46788)
0811 06:23:57.146711 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.146815 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 06:23:57.147058 14654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [term 3 FOLLOWER]: Refusing update from remote peer 68891d8213204e0591f67e622bfc2aff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 06:23:57.147939 15461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 06:23:57.148144 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.147706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46796)
0811 06:23:57.147884 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.147998 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:23:57.148574 14829 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff -> Peer a86a659668f8494db15ce16bc2d7a1c4 (127.7.4.190:35353): Couldn't send request to peer a86a659668f8494db15ce16bc2d7a1c4. Status: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:23:57.150698 15465 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68891d8213204e0591f67e622bfc2aff. Latest consensus state: current_term: 3 leader_uuid: "68891d8213204e0591f67e622bfc2aff" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:57.150974 15465 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:57.152448 15458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "68891d8213204e0591f67e622bfc2aff" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:57.152820 15458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff [sys.catalog]: This master's current role is: LEADER
I20260811 06:23:57.153172 15466 mvcc.cc:205] Tried to move back new op lower bound from 7317214974547206144 to 7317214974396534784. Current Snapshot: MvccSnapshot[applied={T|T < 7317214974547206144}]
I20260811 06:23:57.154222 15465 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "68891d8213204e0591f67e622bfc2aff" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "a86a659668f8494db15ce16bc2d7a1c4" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 35353 } } peers { permanent_uuid: "fd0fca5968034acd953d7416d6009419" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "68891d8213204e0591f67e622bfc2aff" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 39139 } attrs { promote: false } } }
I20260811 06:23:57.154480 15465 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0fca5968034acd953d7416d6009419 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:23:57.175364 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.161330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:36963 (local address 127.0.0.1:52484)
0811 06:23:57.168994 (+ 7664us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.169011 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:57.169142 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:57.169504 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:57.169508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:57.169536 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:57.169824 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.169833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.171156 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.171160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.172935 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.172948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.174152 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.174161 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.174435 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.174493 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 06:23:57.174609 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:23:57.174796 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7525,"thread_start_us":335,"threads_started":1}
I20260811 06:23:57.176093 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.161742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.7.4.130:36963)
0811 06:23:57.167676 (+ 5934us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.167681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.169095 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 06:23:57.169292 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.169296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.169372 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.169476 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:57.169971 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.170992 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.173089 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.173944 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.174070 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.174175 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.174655 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.174812 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5822,"thread_start_us":5715,"threads_started":1}
I20260811 06:23:57.176148 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.161081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46579 (local address 127.0.0.1:40246)
0811 06:23:57.161926 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.161947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:57.162062 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:57.168284 (+ 6222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:57.168289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:57.168316 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:57.168784 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.168803 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.170697 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.170703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.173677 (+ 2974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.173695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.175329 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.175338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.175532 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.175569 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:23:57.175665 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:23:57.175795 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":7580,"threads_started":1}
I20260811 06:23:57.176666 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.161413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.7.4.131:46579)
0811 06:23:57.167995 (+ 6582us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.168000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.168016 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:23:57.168079 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.168083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.168140 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.168259 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:57.169076 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.170477 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.173880 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.174907 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.175036 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.175142 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.176355 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.176478 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6475,"thread_start_us":6393,"threads_started":1}
I20260811 06:23:57.177126 15139 tablet_service.cc:1467] Processing CreateTablet for tablet c919f39dc1e14e99932dd80632349226 (DEFAULT_TABLE table=test-table [id=f7f370f2871e456a89d92162cae27bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:57.177711 15139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c919f39dc1e14e99932dd80632349226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:57.177680 15294 tablet_service.cc:1467] Processing CreateTablet for tablet c919f39dc1e14e99932dd80632349226 (DEFAULT_TABLE table=test-table [id=f7f370f2871e456a89d92162cae27bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:57.178105 15294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c919f39dc1e14e99932dd80632349226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:57.184792 15478 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Bootstrap starting.
I20260811 06:23:57.185819 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.160459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:37319 (local address 127.0.0.1:48202)
0811 06:23:57.160603 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.160624 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:23:57.160748 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:23:57.176781 (+ 16033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:23:57.176786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:23:57.176813 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:23:57.177075 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.177088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.180884 (+ 3796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.180890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.182854 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:23:57.182871 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.185250 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.185255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:23:57.185375 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.185398 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:23:57.185451 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 06:23:57.185534 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 06:23:57.186715 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.160851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.7.4.129:37319)
0811 06:23:57.176431 (+ 15580us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.176437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.176457 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:23:57.176532 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.176537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.176612 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.179464 (+ 2852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:23:57.179663 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.180722 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.183079 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.185030 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.186110 (+ 1080us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.186231 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.186327 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.186441 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15447,"thread_start_us":18488,"threads_started":1}
I20260811 06:23:57.187391 15478 tablet_bootstrap.cc:654] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:57.187579 14985 tablet_service.cc:1467] Processing CreateTablet for tablet c919f39dc1e14e99932dd80632349226 (DEFAULT_TABLE table=test-table [id=f7f370f2871e456a89d92162cae27bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:23:57.187932 15477 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Bootstrap starting.
I20260811 06:23:57.188037 14985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c919f39dc1e14e99932dd80632349226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:23:57.189491 15478 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: No bootstrap required, opened a new log
I20260811 06:23:57.189653 15478 ts_tablet_manager.cc:1397] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 06:23:57.189751 15477 tablet_bootstrap.cc:654] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:57.190613 15478 raft_consensus.cc:348] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.191210 15478 raft_consensus.cc:374] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:57.191350 15478 raft_consensus.cc:729] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84676b31ab348beb6591e1c6c605573, State: Initialized, Role: FOLLOWER
I20260811 06:23:57.191612 15478 consensus_queue.cc:260] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.192431 15478 ts_tablet_manager.cc:1428] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:57.195775 15362 heartbeater.cc:503] Master 127.7.4.188:39139 was elected leader, sending a full tablet report...
I20260811 06:23:57.203477 15477 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: No bootstrap required, opened a new log
I20260811 06:23:57.203679 15477 ts_tablet_manager.cc:1397] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260811 06:23:57.204751 15477 raft_consensus.cc:348] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.205071 15477 raft_consensus.cc:374] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:57.205191 15477 raft_consensus.cc:729] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bdf66c5d574b17a662db2e895a72b7, State: Initialized, Role: FOLLOWER
I20260811 06:23:57.205497 15477 consensus_queue.cc:260] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.206570 15477 ts_tablet_manager.cc:1428] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:23:57.209074 15208 heartbeater.cc:503] Master 127.7.4.188:39139 was elected leader, sending a full tablet report...
I20260811 06:23:57.216389 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.193347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:52007)
0811 06:23:57.215782 (+ 22435us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.215967 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22236,"thread_start_us":22061,"threads_started":1}
I20260811 06:23:57.216424 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.207220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:56189)
0811 06:23:57.216074 (+ 8854us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.216187 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8735,"thread_start_us":8577,"threads_started":1}
I20260811 06:23:57.217504 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.217135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:42861)
0811 06:23:57.217286 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.217370 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 06:23:57.217669 15486 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Bootstrap starting.
I20260811 06:23:57.217722 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.217269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:41459)
0811 06:23:57.217488 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.217571 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 06:23:57.219259 15486 tablet_bootstrap.cc:654] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Neither blocks nor log segments found. Creating new log.
I20260811 06:23:57.221132 15486 tablet_bootstrap.cc:492] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: No bootstrap required, opened a new log
I20260811 06:23:57.221297 15486 ts_tablet_manager.cc:1397] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:23:57.221990 15486 raft_consensus.cc:348] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.222208 15486 raft_consensus.cc:374] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:23:57.222319 15486 raft_consensus.cc:729] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124c93e3f1e9482eaf775e723d857159, State: Initialized, Role: FOLLOWER
I20260811 06:23:57.222551 15486 consensus_queue.cc:260] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.223388 15486 ts_tablet_manager.cc:1428] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:23:57.225513 15057 heartbeater.cc:503] Master 127.7.4.188:39139 was elected leader, sending a full tablet report...
I20260811 06:23:57.231365 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.224334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:55995)
0811 06:23:57.230977 (+ 6643us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.231096 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6529,"thread_start_us":6373,"threads_started":1}
I20260811 06:23:57.397421 15482 raft_consensus.cc:482] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:23:57.397646 15482 raft_consensus.cc:504] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.398360 15482 leader_election.cc:288] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319), 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:57.398952 15009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c919f39dc1e14e99932dd80632349226" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124c93e3f1e9482eaf775e723d857159" is_pre_election: true
I20260811 06:23:57.399019 15160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c919f39dc1e14e99932dd80632349226" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdf66c5d574b17a662db2e895a72b7" is_pre_election: true
I20260811 06:23:57.399269 15009 raft_consensus.cc:2454] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 0.
I20260811 06:23:57.399343 15160 raft_consensus.cc:2454] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 0.
I20260811 06:23:57.399847 15243 leader_election.cc:302] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64bdf66c5d574b17a662db2e895a72b7, b84676b31ab348beb6591e1c6c605573; no voters:
I20260811 06:23:57.400367 15482 raft_consensus.cc:2781] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:23:57.400475 15482 raft_consensus.cc:482] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:23:57.400554 15482 raft_consensus.cc:3037] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:57.403569 15482 raft_consensus.cc:504] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.404148 15482 leader_election.cc:288] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 election: Requested vote from peers 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319), 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963)
I20260811 06:23:57.404678 15009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c919f39dc1e14e99932dd80632349226" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124c93e3f1e9482eaf775e723d857159"
I20260811 06:23:57.404697 15160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c919f39dc1e14e99932dd80632349226" candidate_uuid: "b84676b31ab348beb6591e1c6c605573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdf66c5d574b17a662db2e895a72b7"
I20260811 06:23:57.404832 15009 raft_consensus.cc:3037] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:57.404891 15160 raft_consensus.cc:3037] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:23:57.406898 15009 raft_consensus.cc:2454] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 1.
I20260811 06:23:57.407344 15244 leader_election.cc:302] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 124c93e3f1e9482eaf775e723d857159, b84676b31ab348beb6591e1c6c605573; no voters:
I20260811 06:23:57.407653 15482 raft_consensus.cc:2781] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:23:57.407712 15160 raft_consensus.cc:2454] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84676b31ab348beb6591e1c6c605573 in term 1.
I20260811 06:23:57.408237 15482 raft_consensus.cc:686] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 1 LEADER]: Becoming Leader. State: Replica: b84676b31ab348beb6591e1c6c605573, State: Running, Role: LEADER
I20260811 06:23:57.408565 15482 consensus_queue.cc:237] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } }
I20260811 06:23:57.409299 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.408816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:41251)
0811 06:23:57.409000 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.409120 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:23:57.411401 14845 catalog_manager.cc:5495] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 reported cstate change: term changed from 0 to 1, leader changed from <none> to b84676b31ab348beb6591e1c6c605573 (127.7.4.131). New cstate: current_term: 1 leader_uuid: "b84676b31ab348beb6591e1c6c605573" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b84676b31ab348beb6591e1c6c605573" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 } health_report { overall_health: UNKNOWN } } }
I20260811 06:23:57.464319 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.459991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.7.4.131:46579)
0811 06:23:57.460181 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.460186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.460203 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:23:57.460287 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.460291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.460359 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.460432 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:23:57.460735 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.461447 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.462713 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.463124 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.463244 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.463581 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:23:57.463677 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:23:57.463746 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.463867 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.463927 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:23:57.467543 15009 raft_consensus.cc:1261] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Refusing update from remote peer b84676b31ab348beb6591e1c6c605573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:57.467542 15160 raft_consensus.cc:1261] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Refusing update from remote peer b84676b31ab348beb6591e1c6c605573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:23:57.468235 15492 consensus_queue.cc:1035] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "124c93e3f1e9482eaf775e723d857159" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:57.468580 15482 consensus_queue.cc:1035] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bdf66c5d574b17a662db2e895a72b7" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:23:57.470753 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.470198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:50281)
0811 06:23:57.470413 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.470519 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 06:23:57.471230 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.470826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.129:47731)
0811 06:23:57.470997 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.471083 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:23:57.482195 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.481424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.130:43359)
0811 06:23:57.481861 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.481969 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260811 06:23:57.486475 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.486029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.7.4.131:50505)
0811 06:23:57.486222 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.486316 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 06:23:57.487545 15496 mvcc.cc:205] Tried to move back new op lower bound from 7317214975860781056 to 7317214975628132352. Current Snapshot: MvccSnapshot[applied={T|T < 7317214975860781056}]
I20260811 06:23:57.493943 14845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:57.494318 14845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:23:57.501324 14845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68891d8213204e0591f67e622bfc2aff: Sending DeleteTablet for 3 replicas of tablet c919f39dc1e14e99932dd80632349226
I20260811 06:23:57.502457 14985 tablet_service.cc:1514] Processing DeleteTablet for tablet c919f39dc1e14e99932dd80632349226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:57 UTC) from {username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:48202
I20260811 06:23:57.502583 15139 tablet_service.cc:1514] Processing DeleteTablet for tablet c919f39dc1e14e99932dd80632349226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:57 UTC) from {username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:52484
I20260811 06:23:57.502846 15294 tablet_service.cc:1514] Processing DeleteTablet for tablet c919f39dc1e14e99932dd80632349226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:23:57 UTC) from {username='oryx', principal='oryx/127.7.4.188@KRBTEST.COM'} at 127.0.0.1:40246
I20260811 06:23:57.503314 15502 tablet_replica.cc:330] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: stopping tablet replica
I20260811 06:23:57.504079 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14816
I20260811 06:23:57.510664 15503 tablet_replica.cc:330] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: stopping tablet replica
I20260811 06:23:57.510942 15503 raft_consensus.cc:2229] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:23:57.511315 15503 raft_consensus.cc:2258] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:57.513171 15503 ts_tablet_manager.cc:1905] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:57.514111 15504 tablet_replica.cc:330] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: stopping tablet replica
I20260811 06:23:57.514317 15504 raft_consensus.cc:2229] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:23:57.514446 15502 raft_consensus.cc:2229] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 06:23:57.514588 15492 consensus_queue.cc:1159] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "124c93e3f1e9482eaf775e723d857159" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260811 06:23:57.515103 15504 raft_consensus.cc:2258] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:57.515384 15502 pending_rounds.cc:62] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Trying to abort 2 pending ops.
I20260811 06:23:57.515578 15502 pending_rounds.cc:68] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317214975860781056 op_type: WRITE_OP write_request { tablet_id: "c919f39dc1e14e99932dd80632349226" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\251\202\353\323\006\"8\n\ntest-admin\022*\n f7f370f2871e456a89d92162cae27bd9\020\001\030\001 \001(\001" signature: "<redacted>""@h\274\377\211\364e\277R\246\265\377U\214%\277\221\033]\250\275\202Y[l\244g\361\274\r\333e\031R\203-N\272\341\210x\211\t\364\227l\037\307\023.\345_\357\265\343\315\2226$\363\221\252\333u" signing_key_seq_num: 0 } } request_id { client_id: "36e60d2d125a4a5f9cd3044d18116b42" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260811 06:23:57.516240 15504 ts_tablet_manager.cc:1905] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:57.516716 15502 pending_rounds.cc:68] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317214975628132352 op_type: NO_OP noop_request { }
I20260811 06:23:57.516922 15502 raft_consensus.cc:2866] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 06:23:57.517102 15502 raft_consensus.cc:2258] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:23:57.518219 15502 ts_tablet_manager.cc:1905] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:23:57.520498 15503 ts_tablet_manager.cc:1918] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:57.520641 15503 log.cc:1196] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/c919f39dc1e14e99932dd80632349226
I20260811 06:23:57.521116 15503 ts_tablet_manager.cc:1939] T c919f39dc1e14e99932dd80632349226 P 64bdf66c5d574b17a662db2e895a72b7: Deleting consensus metadata
I20260811 06:23:57.522277 14829 catalog_manager.cc:4841] TS 64bdf66c5d574b17a662db2e895a72b7 (127.7.4.130:36963): tablet c919f39dc1e14e99932dd80632349226 (table test-table [id=f7f370f2871e456a89d92162cae27bd9]) successfully deleted
I20260811 06:23:57.522776 15504 ts_tablet_manager.cc:1918] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:57.522951 15504 log.cc:1196] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/c919f39dc1e14e99932dd80632349226
I20260811 06:23:57.523496 15504 ts_tablet_manager.cc:1939] T c919f39dc1e14e99932dd80632349226 P b84676b31ab348beb6591e1c6c605573: Deleting consensus metadata
I20260811 06:23:57.524695 15502 ts_tablet_manager.cc:1918] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:23:57.524667 14828 catalog_manager.cc:4841] TS b84676b31ab348beb6591e1c6c605573 (127.7.4.131:46579): tablet c919f39dc1e14e99932dd80632349226 (table test-table [id=f7f370f2871e456a89d92162cae27bd9]) successfully deleted
I20260811 06:23:57.524823 15502 log.cc:1196] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/c919f39dc1e14e99932dd80632349226
I20260811 06:23:57.525308 15502 ts_tablet_manager.cc:1939] T c919f39dc1e14e99932dd80632349226 P 124c93e3f1e9482eaf775e723d857159: Deleting consensus metadata
I20260811 06:23:57.526558 14830 catalog_manager.cc:4841] TS 124c93e3f1e9482eaf775e723d857159 (127.7.4.129:37319): tablet c919f39dc1e14e99932dd80632349226 (table test-table [id=f7f370f2871e456a89d92162cae27bd9]) successfully deleted
I20260811 06:23:57.534617 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.505421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.7.4.129:37319)
0811 06:23:57.505666 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.505676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.519619 (+ 13943us) server_negotiation.cc:407] Connection header received
0811 06:23:57.519914 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.519920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.520009 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.520105 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:57.522412 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.523631 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.525248 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.525851 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.525990 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.528670 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:57.528684 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:57.528700 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:57.528731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:57.531747 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.532437 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.532444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.532446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.532575 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.532915 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.532918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.532920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.533531 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:57.533841 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.534125 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.534359 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:23:57.535356 15052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:57.548188 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.547598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46804)
0811 06:23:57.547829 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.547961 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:23:57.549372 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.548911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46808)
0811 06:23:57.549080 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:57.549184 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:23:57.635066 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14816
I20260811 06:23:57.659785 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15072
I20260811 06:23:57.673095 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.660519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.7.4.130:36963)
0811 06:23:57.660737 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.660745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.660769 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:23:57.660985 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.660989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.661056 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.661163 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:57.662464 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.663187 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.664467 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.664905 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.665065 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.667552 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:57.667571 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:57.667587 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:57.667618 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:57.670249 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.670919 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.670928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.670932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.671087 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.671543 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.671549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.671552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.671921 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:57.672247 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.672542 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.672794 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:23:57.673775 15203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:57.767428 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15072
I20260811 06:23:57.789784 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15223
I20260811 06:23:57.802526 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:57.790536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.7.4.131:46579)
0811 06:23:57.790745 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:23:57.790750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:23:57.790769 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:23:57.790908 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:23:57.790912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:23:57.790977 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:23:57.791060 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:23:57.792484 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.793166 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.794324 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:23:57.794702 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:23:57.794792 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:23:57.797195 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:23:57.797211 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:23:57.797227 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:23:57.797265 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:23:57.800226 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.800803 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.800810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.800813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.800934 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:23:57.801251 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:23:57.801254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:23:57.801256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:23:57.801563 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:23:57.801792 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:23:57.802005 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 06:23:57.802163 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:23:57.803234 15357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:57.924933 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15223
I20260811 06:23:57.952195 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14492
I20260811 06:23:57.953244 14676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:58.008049 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:58.007535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46812)
0811 06:23:58.007757 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:58.007874 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:23:58.009106 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:23:58.008729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:35353 (local address 127.0.0.1:46824)
0811 06:23:58.008881 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:23:58.008966 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:35353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:23:58.072365 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14492
I20260811 06:23:58.097635 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 14596
I20260811 06:23:58.098735 14879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:23:58.224395 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 14596
2026-08-11T06:23:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12022 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:23:58 dist-test-slave-07c2 krb5kdc[15514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:23:58 dist-test-slave-07c2 krb5kdc[15514](info): set up 2 sockets
Aug 11 06:23:58 dist-test-slave-07c2 krb5kdc[15514](info): commencing operation
krb5kdc: starting...
W20260811 06:23:59.395892 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:23:59 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:23:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:23:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab.
Entry for principal oryx/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab.
I20260811 06:23:59.672875 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45059
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab
--principal=oryx/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:23:59.996201 15527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:23:59.996541 15527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:23:59.996706 15527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:00.003799 15527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:00.003937 15527 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:24:00.004034 15527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:00.004150 15527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:00.004284 15527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:00.004376 15527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:00.014938 15527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45059
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.190.keytab
--principal=oryx/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:00.016343 15527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:00.017705 15527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:00.030624 15535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:00.030578 15532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:00.030577 15533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:00.031234 15527 server_base.cc:1034] running on GCE node
I20260811 06:24:00.032133 15527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:00.033489 15527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:00.034710 15527 hybrid_clock.cc:648] HybridClock initialized: now 1786429440034641 us; error 35 us; skew 500 ppm
I20260811 06:24:00.034993 15527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:00 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429440, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:00.038245 15527 init.cc:377] Logged in from keytab as oryx/127.7.4.190@KRBTEST.COM (short username oryx)
I20260811 06:24:00.039736 15527 webserver.cc:460] Webserver started at http://127.7.4.190:38467/ using document root <none> and password file <none>
I20260811 06:24:00.040393 15527 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:00.040480 15527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:00.040748 15527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:00.043702 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "077242adbd244cbeb8404a792a04e12f"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "428ace6787a293640756faf44d16e777"
server_key_iv: "f52cc826bf3a87e4b85031e197994411"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.044451 15527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "077242adbd244cbeb8404a792a04e12f"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "428ace6787a293640756faf44d16e777"
server_key_iv: "f52cc826bf3a87e4b85031e197994411"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.049456 15527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:24:00.052876 15542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:00.054207 15527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:24:00.054407 15527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "077242adbd244cbeb8404a792a04e12f"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "428ace6787a293640756faf44d16e777"
server_key_iv: "f52cc826bf3a87e4b85031e197994411"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.054641 15527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:00.072047 15527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:00.076067 15527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:00.076340 15527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:00.098686 15527 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45059
I20260811 06:24:00.098717 15610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45059 every 8 connection(s)
I20260811 06:24:00.100097 15527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:00.103946 15611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:00.104422 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15527
I20260811 06:24:00.104660 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:24:00.105542 7186 external_mini_cluster.cc:1428] Setting key 68a0e44dad88b94e2d7cd0de673ccd5d
I20260811 06:24:00.118063 15611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Bootstrap starting.
Aug 11 06:24:00 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:00.121753 15611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:00.122816 15611 log.cc:824] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:00.125707 15611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: No bootstrap required, opened a new log
I20260811 06:24:00.129004 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:00.107683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.7.4.190:45059)
0811 06:24:00.108558 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:00.108569 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:00.108608 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:24:00.110428 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:00.110450 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:00.110839 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:00.111173 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:00.112903 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.114133 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.115978 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.116612 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.116746 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:00.121420 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:00.121428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:00.121442 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:00.121480 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:00.124991 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.125668 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.125675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.125681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.125894 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.126294 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.126300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.126308 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.126791 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:00.127126 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:00.127579 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:24:00.127995 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":407,"threads_started":1}
I20260811 06:24:00.133760 15611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } }
I20260811 06:24:00.134217 15611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:00.134336 15611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 077242adbd244cbeb8404a792a04e12f, State: Initialized, Role: FOLLOWER
I20260811 06:24:00.134879 15611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } }
I20260811 06:24:00.135123 15611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:00.135269 15611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:00.135495 15611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:00.137794 15611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } }
I20260811 06:24:00.138336 15611 leader_election.cc:302] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 077242adbd244cbeb8404a792a04e12f; no voters:
I20260811 06:24:00.138990 15611 leader_election.cc:288] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:24:00.139160 15616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:00.140688 15616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [term 1 LEADER]: Becoming Leader. State: Replica: 077242adbd244cbeb8404a792a04e12f, State: Running, Role: LEADER
I20260811 06:24:00.141232 15611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:00.141206 15616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } }
I20260811 06:24:00.144207 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 077242adbd244cbeb8404a792a04e12f. Latest consensus state: current_term: 1 leader_uuid: "077242adbd244cbeb8404a792a04e12f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } } }
I20260811 06:24:00.144699 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:00.144236 15617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "077242adbd244cbeb8404a792a04e12f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "077242adbd244cbeb8404a792a04e12f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45059 } } }
I20260811 06:24:00.144969 15617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:00.145751 15624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:00.152384 15624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:00.162001 15624 catalog_manager.cc:1347] Generated new cluster ID: 69c745611a1e4698ae7c929e63ceac16
I20260811 06:24:00.162153 15624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:00.188098 15624 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:24:00.189474 15624 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:00.199807 15624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Generated new TSK 0
I20260811 06:24:00.200564 15624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
I20260811 06:24:00.266865 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45059
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:00.615309 15639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:00.615584 15639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:00.615684 15639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:00.622720 15639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:00.622838 15639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:00.622920 15639 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:24:00.622999 15639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:00.632517 15639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:00.633680 15639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:00.634879 15639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:00.647909 15644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:00.647964 15645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:00.648023 15647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:00.648965 15639 server_base.cc:1034] running on GCE node
I20260811 06:24:00.649573 15639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:00.650477 15639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:00.651718 15639 hybrid_clock.cc:648] HybridClock initialized: now 1786429440651641 us; error 48 us; skew 500 ppm
I20260811 06:24:00.652004 15639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:00 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429440, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:00.655205 15639 init.cc:377] Logged in from keytab as oryx/127.7.4.129@KRBTEST.COM (short username oryx)
I20260811 06:24:00.656566 15639 webserver.cc:460] Webserver started at http://127.7.4.129:44231/ using document root <none> and password file <none>
I20260811 06:24:00.657240 15639 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:00.657371 15639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:00.657687 15639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:00.660648 15639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "cb139d9ca529467a9f551b7b051065a2"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "4048281808410fe49208c2d7991a76d8"
server_key_iv: "b89cad20467ec5763fd7cc1ec1b54066"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.661433 15639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "cb139d9ca529467a9f551b7b051065a2"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "4048281808410fe49208c2d7991a76d8"
server_key_iv: "b89cad20467ec5763fd7cc1ec1b54066"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.666484 15639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:24:00.669768 15655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:00.670900 15639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:00.671070 15639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "cb139d9ca529467a9f551b7b051065a2"
format_stamp: "Formatted at 2026-08-11 06:24:00 on dist-test-slave-07c2"
server_key: "4048281808410fe49208c2d7991a76d8"
server_key_iv: "b89cad20467ec5763fd7cc1ec1b54066"
server_key_version: "encryptionkey@0"
I20260811 06:24:00.671293 15639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:00.684717 15639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:00.688941 15639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:00.689224 15639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:00.691852 15639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:00.691952 15639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:00.692092 15639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:00.692183 15639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:24:00 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429440, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:00.715662 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:00.692371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:49677 (local address 127.7.4.190:45059)
0811 06:24:00.693247 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:00.693252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:00.693913 (+ 661us) server_negotiation.cc:407] Connection header received
0811 06:24:00.695917 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:00.695922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:00.696015 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:00.696138 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:00.698249 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.699263 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.702618 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.703240 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.703425 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:00.708728 (+ 5303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:00.708734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:00.708737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:00.708769 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:00.712106 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.713087 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.713094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.713099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.713255 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.713970 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.713974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.713976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.714236 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:00.714389 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:00.715050 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 06:24:00.715269 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":654,"threads_started":1}
I20260811 06:24:00.716689 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:00.692963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.129:49677)
0811 06:24:00.693759 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:00.693794 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:00.695641 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:00.696403 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:00.696417 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:00.697191 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:00.698075 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:00.698095 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.699438 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.699447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:00.702445 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:00.702462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.703858 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.703867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:00.704121 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:00.705011 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:00.705034 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:00.708412 (+ 3378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:00.712262 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:00.712270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:00.712282 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:00.712889 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:00.713397 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:00.713400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:00.713402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:00.713820 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:00.714389 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:00.714398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:00.714698 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 06:24:00.714957 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:24:00.715480 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":361,"threads_started":1}
I20260811 06:24:00.739059 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:00.723477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45087 (local address 127.7.4.190:45059)
0811 06:24:00.723699 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:00.723705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:00.724604 (+ 899us) server_negotiation.cc:407] Connection header received
0811 06:24:00.724715 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:00.724723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:00.724838 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:00.724963 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:00.725781 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.727112 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.729080 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.729856 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.730009 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:00.732391 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:00.732401 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:00.732408 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:00.732456 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:00.735925 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.736942 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.736946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.736948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.737067 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:00.737472 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:00.737478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:00.737481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:00.737872 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:00.738029 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:00.738452 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 06:24:00.738668 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:24:00.739187 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:00.723585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.129:45087)
0811 06:24:00.724497 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:00.724513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:00.724616 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:00.724974 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:00.724978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:00.725381 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:00.725647 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:00.725655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.727250 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.727253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:00.728877 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:00.728892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:00.730079 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:00.730087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:00.730301 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:00.730917 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:00.730946 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:00.732174 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:00.736114 (+ 3940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:00.736121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:00.736125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:00.736754 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:00.737194 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:00.737199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:00.737202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:00.737346 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:00.737993 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:00.738001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:00.738174 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 06:24:00.738309 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 06:24:00.738667 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":594,"threads_started":1}
I20260811 06:24:00.746951 15639 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:42947
I20260811 06:24:00.746980 15785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:42947 every 8 connection(s)
I20260811 06:24:00.748222 15639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:00.750258 15787 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45059
I20260811 06:24:00.750478 15787 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:00.751104 15787 heartbeater.cc:511] Master 127.7.4.190:45059 requested a full tablet report, sending...
I20260811 06:24:00.753054 15562 ts_manager.cc:194] Registered new tserver with Master: cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947)
I20260811 06:24:00.754637 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15639
I20260811 06:24:00.754730 15562 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:49677
I20260811 06:24:00.754877 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:24:00.755751 7186 external_mini_cluster.cc:1428] Setting key 6a620232226b25ceb822e8fdb3305cf2
WARNING: no policy specified for oryx/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
I20260811 06:24:00.811445 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45059
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:01.141772 15792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:01.142102 15792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:01.142292 15792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:01.149646 15792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:01.149799 15792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:01.149946 15792 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:24:01.150106 15792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:01.160234 15792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:01.161613 15792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:01.162987 15792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:01.176759 15800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:01.176709 15798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:01.176789 15797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:01.177649 15792 server_base.cc:1034] running on GCE node
I20260811 06:24:01.178205 15792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:01.179088 15792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:01.180330 15792 hybrid_clock.cc:648] HybridClock initialized: now 1786429441180253 us; error 50 us; skew 500 ppm
I20260811 06:24:01.180637 15792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:01.183808 15792 init.cc:377] Logged in from keytab as oryx/127.7.4.130@KRBTEST.COM (short username oryx)
I20260811 06:24:01.185213 15792 webserver.cc:460] Webserver started at http://127.7.4.130:40711/ using document root <none> and password file <none>
I20260811 06:24:01.185904 15792 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:01.186036 15792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:01.186363 15792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:01.189325 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "5dfdeac00cc47502fafcf306e3339ad3"
server_key_iv: "9733f5089ab0fe518e69f612218cd50a"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.190124 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "5dfdeac00cc47502fafcf306e3339ad3"
server_key_iv: "9733f5089ab0fe518e69f612218cd50a"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.194995 15792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 06:24:01.198403 15807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:01.199682 15792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:01.199860 15792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "5dfdeac00cc47502fafcf306e3339ad3"
server_key_iv: "9733f5089ab0fe518e69f612218cd50a"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.200086 15792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:01.216655 15792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:01.221282 15792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:01.221570 15792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:01.224344 15792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:01.224514 15792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:24:01.224673 15792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:01.224793 15792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:01.248194 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.224827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:35137 (local address 127.7.4.190:45059)
0811 06:24:01.225039 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.225045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.226353 (+ 1308us) server_negotiation.cc:407] Connection header received
0811 06:24:01.228701 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.228707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.228800 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.228919 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.231104 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.232142 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.234673 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.235332 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.235450 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.240552 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:01.240557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:01.240560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:01.240592 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:01.244055 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.245119 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.245136 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.245141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.245288 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.245956 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.245963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.245966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.246289 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:01.246456 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.247590 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.247816 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:24:01.249946 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.225244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.130:35137)
0811 06:24:01.226199 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.226248 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.228382 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.229187 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.229202 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.229998 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:01.230925 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.230945 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.232321 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.232328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.234527 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.234547 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.235580 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.235588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.236000 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.236776 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:01.236816 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:01.240338 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:01.244247 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.244258 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.244280 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.244979 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.245438 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.245444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.245452 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.245818 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.246554 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:01.246573 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:01.247104 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.247509 (+ 405us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.248113 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":461,"threads_started":1}
I20260811 06:24:01.271168 15792 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:33975
I20260811 06:24:01.271193 15939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:33975 every 8 connection(s)
I20260811 06:24:01.272400 15792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:01.272773 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.257793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:45655 (local address 127.7.4.190:45059)
0811 06:24:01.257949 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.257953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.258759 (+ 806us) server_negotiation.cc:407] Connection header received
0811 06:24:01.258934 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.258939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.259011 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.259107 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.260575 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.261603 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.263057 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.263890 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.264047 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.266402 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:01.266409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:01.266415 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:01.266457 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:01.269755 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.270643 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.270650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.270659 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.270795 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.271190 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.271197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.271202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.271628 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:01.271792 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.272216 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.272428 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:24:01.272917 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.257959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.130:45655)
0811 06:24:01.258657 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.258672 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.258817 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.259135 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.259140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.259875 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:01.260381 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.260398 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.261803 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.261810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.262885 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.262900 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.264116 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.264125 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.264379 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.265034 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:01.265063 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:01.266145 (+ 1082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:01.269888 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.269894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.269898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.270514 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.270915 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.270920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.270924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.271072 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.271758 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:01.271762 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:01.271942 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.272112 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.272454 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":382,"threads_started":1}
I20260811 06:24:01.275257 15941 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45059
I20260811 06:24:01.275506 15941 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:01.276352 15941 heartbeater.cc:511] Master 127.7.4.190:45059 requested a full tablet report, sending...
I20260811 06:24:01.277608 15562 ts_manager.cc:194] Registered new tserver with Master: 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975)
I20260811 06:24:01.277659 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15792
I20260811 06:24:01.277866 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:24:01.278297 7186 external_mini_cluster.cc:1428] Setting key 77d7c0ea26ee5f28d0d6d92cc919b0f9
I20260811 06:24:01.278494 15562 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:35137
WARNING: no policy specified for oryx/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
I20260811 06:24:01.332545 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45059
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:35393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:01.668963 15946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:01.669210 15946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:01.669374 15946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:01.676633 15946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:01.676724 15946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:01.676884 15946 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:24:01.676998 15946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:01.686645 15946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.4.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:01.687896 15946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:01.689194 15946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:01.702126 15951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:01.702088 15952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:01.702172 15954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:01.702814 15946 server_base.cc:1034] running on GCE node
I20260811 06:24:01.703467 15946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:01.704372 15946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:01.705588 15946 hybrid_clock.cc:648] HybridClock initialized: now 1786429441705534 us; error 33 us; skew 500 ppm
I20260811 06:24:01.705901 15946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:01.709069 15946 init.cc:377] Logged in from keytab as oryx/127.7.4.131@KRBTEST.COM (short username oryx)
I20260811 06:24:01.710480 15946 webserver.cc:460] Webserver started at http://127.7.4.131:45157/ using document root <none> and password file <none>
I20260811 06:24:01.711123 15946 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:01.711259 15946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:01.711617 15946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:01.714512 15946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "a7ce06edb00044f0a0d29b66ab73b503"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "efe5178f165eb316f84e035c88ec7805"
server_key_iv: "3a920735c90d512564da9e286304cd49"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.715412 15946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "a7ce06edb00044f0a0d29b66ab73b503"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "efe5178f165eb316f84e035c88ec7805"
server_key_iv: "3a920735c90d512564da9e286304cd49"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.720618 15946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:24:01.724001 15962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:01.725340 15946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:01.725556 15946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "a7ce06edb00044f0a0d29b66ab73b503"
format_stamp: "Formatted at 2026-08-11 06:24:01 on dist-test-slave-07c2"
server_key: "efe5178f165eb316f84e035c88ec7805"
server_key_iv: "3a920735c90d512564da9e286304cd49"
server_key_version: "encryptionkey@0"
I20260811 06:24:01.725805 15946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:01.757431 15787 heartbeater.cc:503] Master 127.7.4.190:45059 was elected leader, sending a full tablet report...
I20260811 06:24:01.768556 15946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:01.772748 15946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:01.773051 15946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:01.776185 15946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:01.776382 15946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:24:01.776620 15946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:01.776777 15946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:01.801441 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.776522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39011 (local address 127.7.4.190:45059)
0811 06:24:01.777018 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.777024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.778392 (+ 1368us) server_negotiation.cc:407] Connection header received
0811 06:24:01.780737 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.780744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.780862 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.780989 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.783536 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.784734 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.787526 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.787963 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.788109 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.793259 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:01.793265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:01.793268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:01.793304 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:01.797039 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.798325 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.798333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.798338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.798539 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.799268 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.799273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.799276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.799599 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:01.799823 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.800994 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.801171 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":243,"threads_started":1}
I20260811 06:24:01.803715 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.777201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.131:39011)
0811 06:24:01.778184 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.778250 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.780388 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.781275 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.781294 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.782236 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:01.783315 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.783346 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.784925 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.784933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.787367 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.787388 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.788280 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.788292 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.788513 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.789478 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:01.789517 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:01.792985 (+ 3468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:01.797219 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.797235 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.797255 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.798126 (+ 871us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.798702 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.798711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.798717 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.799093 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.799807 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:01.799819 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:01.800445 (+ 626us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.800871 (+ 426us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.801619 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":438,"threads_started":1}
I20260811 06:24:01.823261 15946 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46427
I20260811 06:24:01.823261 16092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46427 every 8 connection(s)
I20260811 06:24:01.824606 15946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:01.827584 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 15946
I20260811 06:24:01.827797 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:24:01.828233 7186 external_mini_cluster.cc:1428] Setting key c5cf3da53c74993cd2642976a2c6522f
I20260811 06:24:01.826892 16093 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45059
I20260811 06:24:01.829617 16093 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:01.830034 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.811783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:33857 (local address 127.7.4.190:45059)
0811 06:24:01.811950 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.811954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.813429 (+ 1475us) server_negotiation.cc:407] Connection header received
0811 06:24:01.813700 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.813706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.813814 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.813887 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.815588 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.816862 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.819941 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.820729 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.820884 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.823736 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:01.823741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:01.823744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:01.823775 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:01.827011 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.827780 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.827785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.827789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.827925 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.828303 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.828309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.828317 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.828606 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:01.828770 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.829330 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.829547 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:24:01.830691 16093 heartbeater.cc:511] Master 127.7.4.190:45059 requested a full tablet report, sending...
I20260811 06:24:01.831092 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.811918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.7.4.131:33857)
0811 06:24:01.813254 (+ 1336us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.813294 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.813493 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.814129 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.814139 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.814799 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:01.815361 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.815383 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.817115 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.817124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.819719 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.819736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.821184 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.821191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.821347 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.822112 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:01.822144 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:01.823543 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:01.827117 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.827121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.827124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.827653 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.828041 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:01.828046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:01.828054 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:01.828184 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:01.828737 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:01.828741 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:01.828945 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.829143 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.830516 (+ 1373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"thread_start_us":609,"threads_started":1}
I20260811 06:24:01.832422 15561 ts_manager.cc:194] Registered new tserver with Master: a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:01.833297 15561 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:39011
I20260811 06:24:01.843952 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:24:01.905941 15970 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:01.895371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.7.4.190:45059)
0811 06:24:01.895616 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.895622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.895863 (+ 241us) server_negotiation.cc:407] Connection header received
0811 06:24:01.896087 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.896093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.896173 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.896294 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.897643 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.898371 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.899699 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.900114 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.900272 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.905634 (+ 5362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55178: BlockingRecv error: recv got EOF from 127.0.0.1:55178 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 06:24:01 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:01.921432 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.908807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.7.4.190:45059)
0811 06:24:01.909017 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.909023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.909277 (+ 254us) server_negotiation.cc:407] Connection header received
0811 06:24:01.909389 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.909393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.909461 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.909537 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:01.910554 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.911297 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.912519 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.912944 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.913046 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.916643 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:01.916675 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:01.916678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:01.916712 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:01.919120 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.919807 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.919810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.919813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.919935 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:01.920348 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:01.920351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:01.920353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:01.920602 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:01.920804 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.921038 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.921206 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:24:01.925473 15561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:24:01.932468 15561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:01.958068 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.944749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33975 (local address 127.0.0.1:33976)
0811 06:24:01.946609 (+ 1860us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.946694 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.947029 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.948139 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.948145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.948178 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:01.948676 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.948693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.951230 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.951237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.954224 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.954246 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.956738 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.956748 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.957053 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.957133 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.957286 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.957533 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"thread_start_us":1406,"threads_started":1}
I20260811 06:24:01.959395 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.944981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33976 (local address 127.7.4.130:33975)
0811 06:24:01.945965 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.945975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.946828 (+ 853us) server_negotiation.cc:407] Connection header received
0811 06:24:01.947348 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.947360 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.947645 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.947843 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:01.948906 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.950980 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.954459 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.956169 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.956269 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.956496 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.958731 (+ 2235us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.958973 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":688,"threads_started":1}
I20260811 06:24:01.965371 15865 tablet_service.cc:1467] Processing CreateTablet for tablet cc17a348e97c4aca9db4752d30255884 (DEFAULT_TABLE table=test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:01.966863 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc17a348e97c4aca9db4752d30255884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:01.970999 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.947152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42947 (local address 127.0.0.1:41720)
0811 06:24:01.960737 (+ 13585us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.960753 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.960878 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.961941 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.961949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.961985 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:01.962527 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.962542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.965516 (+ 2974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.965525 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.968208 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.968230 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.970173 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.970179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.970367 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.970413 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.970520 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.970664 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13437,"thread_start_us":13290,"threads_started":1}
I20260811 06:24:01.971063 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.947809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.7.4.129:42947)
0811 06:24:01.960263 (+ 12454us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.960276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.961007 (+ 731us) server_negotiation.cc:407] Connection header received
0811 06:24:01.961122 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.961133 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.961400 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.961668 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:01.962785 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.964912 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.968425 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.969957 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.970100 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.970308 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.970577 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.970761 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12238,"thread_start_us":12045,"threads_started":1}
I20260811 06:24:01.974921 15708 tablet_service.cc:1467] Processing CreateTablet for tablet cc17a348e97c4aca9db4752d30255884 (DEFAULT_TABLE table=test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:01.976100 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.945518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46427 (local address 127.0.0.1:54068)
0811 06:24:01.958348 (+ 12830us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:01.958372 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:01.958506 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:01.969456 (+ 10950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:01.969461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:01.969489 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:01.969712 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.969722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.971748 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.971753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.973371 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:01.973386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.975254 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.975261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:01.975491 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.975523 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:24:01.975612 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:24:01.975776 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12714,"thread_start_us":19705,"threads_started":1}
I20260811 06:24:01.976258 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:01.946219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.7.4.131:46427)
0811 06:24:01.968988 (+ 22769us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:01.968994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:01.969020 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:24:01.969094 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:01.969099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:01.969263 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:01.969471 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:01.969938 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.971603 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.973594 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:01.975066 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:01.975209 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:01.975477 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:01.975698 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 06:24:01.975870 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22610,"thread_start_us":22391,"threads_started":1}
I20260811 06:24:01.976845 15708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc17a348e97c4aca9db4752d30255884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:01.980000 16018 tablet_service.cc:1467] Processing CreateTablet for tablet cc17a348e97c4aca9db4752d30255884 (DEFAULT_TABLE table=test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:01.981451 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc17a348e97c4aca9db4752d30255884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:01.982270 16118 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Bootstrap starting.
I20260811 06:24:01.985920 16118 tablet_bootstrap.cc:654] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:01.987195 16118 log.cc:824] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:01.989876 16121 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Bootstrap starting.
I20260811 06:24:01.990407 16118 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: No bootstrap required, opened a new log
I20260811 06:24:01.990742 16118 ts_tablet_manager.cc:1397] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260811 06:24:01.993614 16121 tablet_bootstrap.cc:654] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:01.994999 16121 log.cc:824] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:01.995635 16122 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Bootstrap starting.
I20260811 06:24:01.999320 16122 tablet_bootstrap.cc:654] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.000631 16122 log.cc:824] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:02.000546 16118 raft_consensus.cc:348] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.001055 16118 raft_consensus.cc:374] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.001188 16118 raft_consensus.cc:729] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.002243 16121 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: No bootstrap required, opened a new log
I20260811 06:24:02.002039 16118 consensus_queue.cc:260] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.002560 16121 ts_tablet_manager.cc:1397] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260811 06:24:02.003456 15941 heartbeater.cc:503] Master 127.7.4.190:45059 was elected leader, sending a full tablet report...
I20260811 06:24:02.006014 16118 ts_tablet_manager.cc:1428] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260811 06:24:02.011190 16122 tablet_bootstrap.cc:492] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: No bootstrap required, opened a new log
I20260811 06:24:02.011528 16122 ts_tablet_manager.cc:1397] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260811 06:24:02.012425 16121 raft_consensus.cc:348] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.012884 16121 raft_consensus.cc:374] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.013000 16121 raft_consensus.cc:729] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.013624 16121 consensus_queue.cc:260] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.015399 16121 ts_tablet_manager.cc:1428] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260811 06:24:02.020452 16122 raft_consensus.cc:348] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.020792 16122 raft_consensus.cc:374] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.020864 16122 raft_consensus.cc:729] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.021401 16122 consensus_queue.cc:260] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.022361 16093 heartbeater.cc:503] Master 127.7.4.190:45059 was elected leader, sending a full tablet report...
I20260811 06:24:02.022820 16122 ts_tablet_manager.cc:1428] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
W20260811 06:24:02.026897 15942 tablet.cc:2369] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:24:02.071950 16126 raft_consensus.cc:482] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:02.072250 16126 raft_consensus.cc:504] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.074005 16126 leader_election.cc:288] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:02.077471 16125 raft_consensus.cc:482] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:02.077780 16125 raft_consensus.cc:504] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
W20260811 06:24:02.078270 16094 tablet.cc:2369] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:24:02.079768 16125 leader_election.cc:288] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:02.083696 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.074857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:40955 (local address 127.7.4.131:46427)
0811 06:24:02.075061 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.075066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.075732 (+ 666us) server_negotiation.cc:407] Connection header received
0811 06:24:02.076016 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.076021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.076104 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.076208 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.076937 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.077863 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.080495 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.081826 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.081961 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.082102 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.083137 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.083341 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:24:02.083917 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.074501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:43511 (local address 127.7.4.130:33975)
0811 06:24:02.074723 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.074732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.075534 (+ 802us) server_negotiation.cc:407] Connection header received
0811 06:24:02.075750 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.075758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.075871 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.075970 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.076653 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.077709 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.080386 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.082141 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.082322 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.082514 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.083323 (+ 809us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.083524 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:24:02.084896 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.074386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33975 (local address 127.7.4.129:43511)
0811 06:24:02.075407 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.075431 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.075578 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.075997 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.076001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.076030 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.076466 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.076480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.077857 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.077866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.080184 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.080202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.082816 (+ 2614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.082825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.083073 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.083123 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.084099 (+ 976us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.084298 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":7,"thread_start_us":533,"threads_started":1}
I20260811 06:24:02.085048 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.074642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46427 (local address 127.7.4.129:40955)
0811 06:24:02.075587 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.075626 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.075838 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.076236 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.076240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.076287 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.076751 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.076765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.078075 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.078081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.080304 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.080320 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.082623 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.082631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.082908 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.082960 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.084363 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.084558 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":586,"threads_started":1}
I20260811 06:24:02.086571 16041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503" is_pre_election: true
I20260811 06:24:02.087225 16041 raft_consensus.cc:2454] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 0.
I20260811 06:24:02.086974 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" is_pre_election: true
I20260811 06:24:02.087702 15885 raft_consensus.cc:2454] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 0.
I20260811 06:24:02.088001 15659 leader_election.cc:302] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:02.088757 16126 raft_consensus.cc:2781] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:02.088919 16126 raft_consensus.cc:482] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:02.089031 16126 raft_consensus.cc:3037] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.092077 16126 raft_consensus.cc:504] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.093272 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
I20260811 06:24:02.093325 16041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503"
I20260811 06:24:02.093506 15885 raft_consensus.cc:3037] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.093520 16041 raft_consensus.cc:3037] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.093962 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.080359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46427 (local address 127.7.4.130:50115)
0811 06:24:02.086388 (+ 6029us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.086405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.086568 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.087061 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.087066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.087121 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.087814 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.087830 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.089038 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.089043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.091095 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.091110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.092729 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.092737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.092905 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.092942 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.093362 (+ 420us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.093507 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5874,"mutex_wait_us":24,"thread_start_us":4892,"threads_started":1}
I20260811 06:24:02.094170 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.081079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38001 (local address 127.7.4.129:42947)
0811 06:24:02.082383 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.082391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.086274 (+ 3883us) server_negotiation.cc:407] Connection header received
0811 06:24:02.086562 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.086567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.086655 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.086728 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.088428 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.089563 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.091963 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.093046 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.093179 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.093270 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.093800 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.093946 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073}
I20260811 06:24:02.094733 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.079985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42947 (local address 127.7.4.130:38001)
0811 06:24:02.086167 (+ 6182us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.086192 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.086391 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.086921 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.086925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.086950 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.088256 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.088270 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.089753 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.089758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.091800 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.091812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.093435 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.093441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.093604 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.093642 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.094333 (+ 691us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.094453 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5470,"mutex_wait_us":56,"thread_start_us":6055,"threads_started":1}
I20260811 06:24:02.095552 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.080786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:50115 (local address 127.7.4.131:46427)
0811 06:24:02.083973 (+ 3187us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.083981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.086484 (+ 2503us) server_negotiation.cc:407] Connection header received
0811 06:24:02.086714 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.086718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.086809 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.086868 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.087980 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.088914 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.091306 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.092490 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.092630 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.092724 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.093115 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.093247 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3065,"thread_start_us":4996,"threads_started":1}
I20260811 06:24:02.095747 16126 leader_election.cc:288] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 election: Requested vote from peers 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:02.096866 15885 raft_consensus.cc:2454] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 1.
I20260811 06:24:02.096897 16041 raft_consensus.cc:2454] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 1.
I20260811 06:24:02.097337 15659 leader_election.cc:302] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:02.097898 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503" is_pre_election: true
I20260811 06:24:02.098337 16040 raft_consensus.cc:2379] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 492fa65b76c54cbf9bb1d8ecbb4d96d1 in current term 1: Already voted for candidate cb139d9ca529467a9f551b7b051065a2 in this term.
I20260811 06:24:02.098667 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc17a348e97c4aca9db4752d30255884" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2" is_pre_election: true
I20260811 06:24:02.098977 16126 raft_consensus.cc:2781] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:02.103436 16126 raft_consensus.cc:686] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Running, Role: LEADER
I20260811 06:24:02.104025 16126 consensus_queue.cc:237] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.109117 15808 leader_election.cc:302] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1; no voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2
I20260811 06:24:02.109629 16125 raft_consensus.cc:2726] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:24:02.109896 15562 catalog_manager.cc:5495] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb139d9ca529467a9f551b7b051065a2 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "cb139d9ca529467a9f551b7b051065a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } health_report { overall_health: UNKNOWN } } }
I20260811 06:24:02.149374 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.143699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.7.4.129:42947)
0811 06:24:02.143901 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.143908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.144022 (+ 114us) server_negotiation.cc:407] Connection header received
0811 06:24:02.144314 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.144318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.144413 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.144568 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:02.144946 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.145682 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.147022 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.147463 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.147602 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.148126 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.148708 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.148867 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.148954 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.149063 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:24:02.158051 16040 raft_consensus.cc:1261] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Refusing update from remote peer cb139d9ca529467a9f551b7b051065a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.158006 15885 raft_consensus.cc:1261] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Refusing update from remote peer cb139d9ca529467a9f551b7b051065a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.159015 16126 consensus_queue.cc:1035] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.159430 16133 consensus_queue.cc:1035] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.171180 16138 mvcc.cc:205] Tried to move back new op lower bound from 7317214995068227584 to 7317214994862448640. Current Snapshot: MvccSnapshot[applied={T|T < 7317214995068227584}]
I20260811 06:24:02.171819 16140 mvcc.cc:205] Tried to move back new op lower bound from 7317214995068227584 to 7317214994862448640. Current Snapshot: MvccSnapshot[applied={T|T < 7317214995068227584}]
I20260811 06:24:02.187685 15561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:02.187983 15561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:02.191823 15561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Sending DeleteTablet for 3 replicas of tablet cc17a348e97c4aca9db4752d30255884
I20260811 06:24:02.193102 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet cc17a348e97c4aca9db4752d30255884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:33976
I20260811 06:24:02.193135 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet cc17a348e97c4aca9db4752d30255884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:54068
I20260811 06:24:02.193897 15708 tablet_service.cc:1514] Processing DeleteTablet for tablet cc17a348e97c4aca9db4752d30255884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:41720
I20260811 06:24:02.195947 15561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:24:02.197741 15561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:02.210148 15863 tablet_service.cc:1467] Processing CreateTablet for tablet 6142f7c9286c44b58ebe162fd82b077f (DEFAULT_TABLE table=test-table [id=2a6411ea5ceb452f8023fd41c7165980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:02.210798 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6142f7c9286c44b58ebe162fd82b077f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.210951 15707 tablet_service.cc:1467] Processing CreateTablet for tablet 6142f7c9286c44b58ebe162fd82b077f (DEFAULT_TABLE table=test-table [id=2a6411ea5ceb452f8023fd41c7165980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:02.211807 15707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6142f7c9286c44b58ebe162fd82b077f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.215803 16016 tablet_service.cc:1467] Processing CreateTablet for tablet 6142f7c9286c44b58ebe162fd82b077f (DEFAULT_TABLE table=test-table [id=2a6411ea5ceb452f8023fd41c7165980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:02.216358 16016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6142f7c9286c44b58ebe162fd82b077f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.220717 16121 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Bootstrap starting.
I20260811 06:24:02.222846 16121 tablet_bootstrap.cc:654] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.225553 16118 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Bootstrap starting.
I20260811 06:24:02.227342 16118 tablet_bootstrap.cc:654] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.230361 16122 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Bootstrap starting.
I20260811 06:24:02.232666 16122 tablet_bootstrap.cc:654] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.236112 16150 tablet_replica.cc:330] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: stopping tablet replica
I20260811 06:24:02.237018 16118 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: No bootstrap required, opened a new log
I20260811 06:24:02.237252 16118 ts_tablet_manager.cc:1397] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260811 06:24:02.238198 16118 raft_consensus.cc:348] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.238567 16118 raft_consensus.cc:374] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.238725 16118 raft_consensus.cc:729] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.239090 16118 consensus_queue.cc:260] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.242678 16118 ts_tablet_manager.cc:1428] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 06:24:02.245740 16149 tablet_replica.cc:330] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: stopping tablet replica
I20260811 06:24:02.240694 16122 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: No bootstrap required, opened a new log
I20260811 06:24:02.246030 16122 ts_tablet_manager.cc:1397] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260811 06:24:02.246938 16122 raft_consensus.cc:348] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.247165 16122 raft_consensus.cc:374] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.247252 16122 raft_consensus.cc:729] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.247556 16122 consensus_queue.cc:260] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.248190 16122 ts_tablet_manager.cc:1428] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:24:02.248970 16150 raft_consensus.cc:2229] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:02.249506 16149 raft_consensus.cc:2229] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:02.250095 16149 raft_consensus.cc:2258] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.250583 16150 raft_consensus.cc:2258] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.253170 16121 tablet_bootstrap.cc:492] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: No bootstrap required, opened a new log
I20260811 06:24:02.253412 16121 ts_tablet_manager.cc:1397] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.011s
I20260811 06:24:02.253500 16151 tablet_replica.cc:330] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: stopping tablet replica
I20260811 06:24:02.254010 16151 raft_consensus.cc:2229] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:24:02.254401 16121 raft_consensus.cc:348] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.254560 16149 ts_tablet_manager.cc:1905] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.254705 16121 raft_consensus.cc:374] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.254792 16150 ts_tablet_manager.cc:1905] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.254866 16121 raft_consensus.cc:729] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.255002 16151 raft_consensus.cc:2258] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.255191 16121 consensus_queue.cc:260] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.255856 16121 ts_tablet_manager.cc:1428] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:02.261909 16149 ts_tablet_manager.cc:1918] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:02.262159 16149 log.cc:1196] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/cc17a348e97c4aca9db4752d30255884
I20260811 06:24:02.262504 16151 ts_tablet_manager.cc:1905] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.262838 16149 ts_tablet_manager.cc:1939] T cc17a348e97c4aca9db4752d30255884 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting consensus metadata
I20260811 06:24:02.263029 16150 ts_tablet_manager.cc:1918] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:02.263226 16150 log.cc:1196] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/cc17a348e97c4aca9db4752d30255884
I20260811 06:24:02.263803 16150 ts_tablet_manager.cc:1939] T cc17a348e97c4aca9db4752d30255884 P a7ce06edb00044f0a0d29b66ab73b503: Deleting consensus metadata
I20260811 06:24:02.264869 15543 catalog_manager.cc:4841] TS 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975): tablet cc17a348e97c4aca9db4752d30255884 (table test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]) successfully deleted
I20260811 06:24:02.265461 15546 catalog_manager.cc:4841] TS a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427): tablet cc17a348e97c4aca9db4752d30255884 (table test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]) successfully deleted
I20260811 06:24:02.268860 16151 ts_tablet_manager.cc:1918] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:02.269083 16151 log.cc:1196] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/cc17a348e97c4aca9db4752d30255884
I20260811 06:24:02.269559 16151 ts_tablet_manager.cc:1939] T cc17a348e97c4aca9db4752d30255884 P cb139d9ca529467a9f551b7b051065a2: Deleting consensus metadata
I20260811 06:24:02.270977 15543 catalog_manager.cc:4841] TS cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947): tablet cc17a348e97c4aca9db4752d30255884 (table test-table [id=0cbfc9832cf54aa9bfab7c6e7b766509]) successfully deleted
I20260811 06:24:02.434284 16127 raft_consensus.cc:482] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:02.434538 16127 raft_consensus.cc:504] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.436463 16127 leader_election.cc:288] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975)
I20260811 06:24:02.444312 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.436801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:46601 (local address 127.7.4.129:42947)
0811 06:24:02.437026 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.437033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.437437 (+ 404us) server_negotiation.cc:407] Connection header received
0811 06:24:02.437780 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.437789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.437928 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.438013 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.438763 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.440016 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.441884 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.443270 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.443426 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.443529 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.443850 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.444015 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:24:02.444334 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.436757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:42947 (local address 127.7.4.131:46601)
0811 06:24:02.437330 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.437353 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.437560 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.438242 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.438245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.438268 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.438583 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.438595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.440168 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.440173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.441694 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.441706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.443454 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.443461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.443643 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.443692 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.443798 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.443938 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":314,"threads_started":1}
I20260811 06:24:02.445117 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.437488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33975 (local address 127.7.4.131:38253)
0811 06:24:02.438232 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.438256 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.438408 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.438787 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.438791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.438815 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:02.439089 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.439098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.440219 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.440225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.442466 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.442483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.444270 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.444283 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.444539 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.444571 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.444663 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.444795 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":489,"threads_started":1}
I20260811 06:24:02.445158 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.437673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:38253 (local address 127.7.4.130:33975)
0811 06:24:02.437851 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.437857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.438391 (+ 534us) server_negotiation.cc:407] Connection header received
0811 06:24:02.438568 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.438573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.438675 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.438776 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:02.439266 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.440054 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.442639 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.443882 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.444002 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.444090 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.444728 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.444883 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:24:02.445384 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6142f7c9286c44b58ebe162fd82b077f" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2" is_pre_election: true
I20260811 06:24:02.445722 15726 raft_consensus.cc:2454] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 0.
I20260811 06:24:02.446409 15963 leader_election.cc:302] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:02.446820 16127 raft_consensus.cc:2781] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:02.446952 16127 raft_consensus.cc:482] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:02.447036 16127 raft_consensus.cc:3037] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.447273 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6142f7c9286c44b58ebe162fd82b077f" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" is_pre_election: true
I20260811 06:24:02.447588 15885 raft_consensus.cc:2454] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 0.
I20260811 06:24:02.450402 16127 raft_consensus.cc:504] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.451123 16127 leader_election.cc:288] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 election: Requested vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975)
I20260811 06:24:02.451623 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6142f7c9286c44b58ebe162fd82b077f" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2"
I20260811 06:24:02.451730 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6142f7c9286c44b58ebe162fd82b077f" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
I20260811 06:24:02.451890 15726 raft_consensus.cc:3037] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.452010 15885 raft_consensus.cc:3037] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.454262 15885 raft_consensus.cc:2454] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 1.
I20260811 06:24:02.454718 15963 leader_election.cc:302] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1, a7ce06edb00044f0a0d29b66ab73b503; no voters:
I20260811 06:24:02.455037 16127 raft_consensus.cc:2781] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:02.455448 15726 raft_consensus.cc:2454] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 1.
I20260811 06:24:02.455597 16127 raft_consensus.cc:686] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 1 LEADER]: Becoming Leader. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Running, Role: LEADER
I20260811 06:24:02.456295 16127 consensus_queue.cc:237] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:02.460116 15562 catalog_manager.cc:5495] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131). New cstate: current_term: 1 leader_uuid: "a7ce06edb00044f0a0d29b66ab73b503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } health_report { overall_health: HEALTHY } } }
W20260811 06:24:02.502912 15789 tablet.cc:2369] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:24:02 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429440, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.190@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:02.559547 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.537849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45059 (local address 127.0.0.1:55186)
0811 06:24:02.538598 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.538624 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.538812 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.539171 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.539175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.539946 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:02.540188 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.540198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.541374 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.541381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.542402 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.542416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.543615 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.543620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.543764 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.549840 (+ 6076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:02.549874 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:02.552406 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:02.556711 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:02.556726 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:02.556732 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:02.557148 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:02.557576 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:02.557579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:02.557581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:02.557738 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:02.558422 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:02.558429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:02.558590 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.558764 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.559072 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":392,"threads_started":1}
I20260811 06:24:02.559628 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.537888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.7.4.190:45059)
0811 06:24:02.538900 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.538906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.538927 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:24:02.538976 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.538979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.539031 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.539144 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:02.540366 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.541223 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.542597 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.543364 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.543517 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.552761 (+ 9244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:02.552774 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:02.552780 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:02.552823 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:02.556550 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:02.557286 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:02.557291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:02.557296 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:02.557457 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:02.557865 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:02.557872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:02.557875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:02.558296 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:02.558461 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.558901 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.559220 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":894,"thread_start_us":728,"threads_started":1}
I20260811 06:24:02.564787 15562 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:55186:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:24:02.565822 15562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:02.573487 15708 tablet_service.cc:1467] Processing CreateTablet for tablet 4505b27579d04a2cbd1687239f286fab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e1be0266c824a27a3ff358b956d4df3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:24:02.574079 15708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4505b27579d04a2cbd1687239f286fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.580478 16121 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: Bootstrap starting.
I20260811 06:24:02.580668 16018 tablet_service.cc:1467] Processing CreateTablet for tablet 4505b27579d04a2cbd1687239f286fab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e1be0266c824a27a3ff358b956d4df3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:24:02.581056 15865 tablet_service.cc:1467] Processing CreateTablet for tablet 4505b27579d04a2cbd1687239f286fab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e1be0266c824a27a3ff358b956d4df3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:24:02.581286 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4505b27579d04a2cbd1687239f286fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.581653 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4505b27579d04a2cbd1687239f286fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:02.582376 16121 tablet_bootstrap.cc:654] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.590914 16118 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Bootstrap starting.
I20260811 06:24:02.592463 16122 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: Bootstrap starting.
I20260811 06:24:02.592798 16118 tablet_bootstrap.cc:654] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.592971 16121 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: No bootstrap required, opened a new log
I20260811 06:24:02.593152 16121 ts_tablet_manager.cc:1397] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260811 06:24:02.593961 16121 raft_consensus.cc:348] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.594162 16122 tablet_bootstrap.cc:654] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:02.594237 16121 raft_consensus.cc:374] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.594310 16121 raft_consensus.cc:729] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.594617 16121 consensus_queue.cc:260] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.595121 16144 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.595539 16144 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.597198 16121 ts_tablet_manager.cc:1428] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 06:24:02.606248 16122 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: No bootstrap required, opened a new log
I20260811 06:24:02.606515 16122 ts_tablet_manager.cc:1397] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260811 06:24:02.606786 16118 tablet_bootstrap.cc:492] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: No bootstrap required, opened a new log
I20260811 06:24:02.607035 16118 ts_tablet_manager.cc:1397] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260811 06:24:02.607472 16122 raft_consensus.cc:348] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.607728 16122 raft_consensus.cc:374] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.607813 16122 raft_consensus.cc:729] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.607920 16118 raft_consensus.cc:348] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.608064 16122 consensus_queue.cc:260] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.608203 16118 raft_consensus.cc:374] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:02.608287 16118 raft_consensus.cc:729] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Initialized, Role: FOLLOWER
I20260811 06:24:02.608512 16127 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.608875 16127 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.609315 16122 ts_tablet_manager.cc:1428] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:24:02.609905 16118 consensus_queue.cc:260] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.610350 16118 ts_tablet_manager.cc:1428] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 06:24:02.610667 16125 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.610962 16125 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.758313 16125 raft_consensus.cc:482] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:02.758536 16125 raft_consensus.cc:504] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.759320 16125 leader_election.cc:288] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:02.759847 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4505b27579d04a2cbd1687239f286fab" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2" is_pre_election: true
I20260811 06:24:02.759886 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4505b27579d04a2cbd1687239f286fab" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503" is_pre_election: true
I20260811 06:24:02.760119 15726 raft_consensus.cc:2454] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492fa65b76c54cbf9bb1d8ecbb4d96d1 in term 0.
I20260811 06:24:02.760124 16040 raft_consensus.cc:2454] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492fa65b76c54cbf9bb1d8ecbb4d96d1 in term 0.
I20260811 06:24:02.760582 15808 leader_election.cc:302] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:02.760967 16125 raft_consensus.cc:2781] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:02.761089 16125 raft_consensus.cc:482] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:02.761195 16125 raft_consensus.cc:3037] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.763350 16125 raft_consensus.cc:504] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.763958 16125 leader_election.cc:288] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 election: Requested vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:02.764460 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4505b27579d04a2cbd1687239f286fab" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2"
I20260811 06:24:02.764492 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4505b27579d04a2cbd1687239f286fab" candidate_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503"
I20260811 06:24:02.764709 15726 raft_consensus.cc:3037] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.764721 16040 raft_consensus.cc:3037] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:02.767740 16040 raft_consensus.cc:2454] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492fa65b76c54cbf9bb1d8ecbb4d96d1 in term 1.
I20260811 06:24:02.767752 15726 raft_consensus.cc:2454] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492fa65b76c54cbf9bb1d8ecbb4d96d1 in term 1.
I20260811 06:24:02.768168 15812 leader_election.cc:302] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1, a7ce06edb00044f0a0d29b66ab73b503; no voters:
I20260811 06:24:02.768451 16125 raft_consensus.cc:2781] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:02.768870 16125 raft_consensus.cc:686] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 LEADER]: Becoming Leader. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Running, Role: LEADER
I20260811 06:24:02.769277 16125 consensus_queue.cc:237] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } }
I20260811 06:24:02.771066 16171 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: TxnStatusTablet state changed. Reason: New leader 492fa65b76c54cbf9bb1d8ecbb4d96d1. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.771366 16171 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:24:02.771960 16174 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:24:02.772218 16174 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:24:02.772866 15561 catalog_manager.cc:5495] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130). New cstate: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } health_report { overall_health: HEALTHY } } }
I20260811 06:24:02.794608 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.788824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33975 (local address 127.0.0.1:33980)
0811 06:24:02.789063 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.789078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.789243 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.789754 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.789759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.789785 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:24:02.789959 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.789968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.791093 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.791098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.792210 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.792223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.792975 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.792981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.793175 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.793231 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.794086 (+ 855us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.794107 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.794196 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.794342 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:24:02.794592 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.788784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.7.4.130:33975)
0811 06:24:02.788956 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.788962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.789148 (+ 186us) server_negotiation.cc:407] Connection header received
0811 06:24:02.789452 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.789456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.789547 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.789720 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:02.790089 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.790945 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.792355 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.792801 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.792941 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.793442 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.793967 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.794126 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.794218 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.794298 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:24:02.799115 16040 raft_consensus.cc:1261] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Refusing update from remote peer 492fa65b76c54cbf9bb1d8ecbb4d96d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.799381 15726 raft_consensus.cc:1261] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Refusing update from remote peer 492fa65b76c54cbf9bb1d8ecbb4d96d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.799906 16171 consensus_queue.cc:1035] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.800282 16125 consensus_queue.cc:1035] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.812077 16144 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: TxnStatusTablet state changed. Reason: New leader 492fa65b76c54cbf9bb1d8ecbb4d96d1. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.812412 16144 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.814582 16177 mvcc.cc:205] Tried to move back new op lower bound from 7317214997698162688 to 7317214997586567168. Current Snapshot: MvccSnapshot[applied={T|T < 7317214997698162688}]
I20260811 06:24:02.816390 16171 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.816725 16171 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:24:02.821548 16127 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: TxnStatusTablet state changed. Reason: New leader 492fa65b76c54cbf9bb1d8ecbb4d96d1. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.821822 16127 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.824524 16144 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.824817 16144 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P cb139d9ca529467a9f551b7b051065a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.824747 16171 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.828074 16127 tablet_replica.cc:439] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } }
I20260811 06:24:02.828419 16127 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P a7ce06edb00044f0a0d29b66ab73b503: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:24:02.831619 16171 tablet_replica.cc:442] T 4505b27579d04a2cbd1687239f286fab P 492fa65b76c54cbf9bb1d8ecbb4d96d1: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:24:02.844472 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.829675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.7.4.131:46427)
0811 06:24:02.830232 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.830238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.835885 (+ 5647us) server_negotiation.cc:407] Connection header received
0811 06:24:02.837366 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.837372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.837464 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.837611 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:02.838190 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.839404 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.841519 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.842265 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.842379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.843048 (+ 669us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.843705 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.843844 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.843987 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.844067 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":265,"threads_started":1}
I20260811 06:24:02.861862 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.855753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:56611 (local address 127.7.4.130:33975)
0811 06:24:02.855954 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.855959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.856483 (+ 524us) server_negotiation.cc:407] Connection header received
0811 06:24:02.856692 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.856697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.856778 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.856877 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:02.857388 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.858306 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.859910 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.860471 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.860635 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.861090 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.861206 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.861322 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.861461 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.861537 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:24:02.861943 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.855748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33975 (local address 127.7.4.131:56611)
0811 06:24:02.856373 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.856389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.856548 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.856892 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.856897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.856909 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:24:02.857221 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.857229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.858440 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.858446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.859766 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.859778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.860684 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.860689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.860844 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.860892 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.861324 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.861337 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.861435 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.861559 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":380,"threads_started":1}
I20260811 06:24:02.871775 15885 raft_consensus.cc:1261] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Refusing update from remote peer a7ce06edb00044f0a0d29b66ab73b503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.871893 15726 raft_consensus.cc:1261] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Refusing update from remote peer a7ce06edb00044f0a0d29b66ab73b503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:02.872409 16157 consensus_queue.cc:1035] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.873018 16127 consensus_queue.cc:1035] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:02.911703 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.905620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:54095 (local address 127.7.4.131:46427)
0811 06:24:02.905804 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:02.905812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:02.906581 (+ 769us) server_negotiation.cc:407] Connection header received
0811 06:24:02.906778 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:02.906784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:02.906856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:02.906948 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:02.907631 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.908361 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.909929 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.910358 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.910506 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.910927 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.911031 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.911125 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:02.911275 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:24:02.911371 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:24:02.912581 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:02.905607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:46427 (local address 127.7.4.130:54095)
0811 06:24:02.906477 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:02.906493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:02.906661 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:02.906971 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:02.906976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:02.906988 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:24:02.907472 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.907484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.908508 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.908513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.909778 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:02.909790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:02.910533 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:02.910539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:02.910698 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:02.910756 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:24:02.911131 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:24:02.911144 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:24:02.911252 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:24:02.911426 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":495,"threads_started":1}
I20260811 06:24:02.960173 15562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:02.960439 15562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:02.963909 15562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Sending DeleteTablet for 3 replicas of tablet 6142f7c9286c44b58ebe162fd82b077f
I20260811 06:24:02.964735 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6142f7c9286c44b58ebe162fd82b077f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:33976
I20260811 06:24:02.964815 15708 tablet_service.cc:1514] Processing DeleteTablet for tablet 6142f7c9286c44b58ebe162fd82b077f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:41720
I20260811 06:24:02.965523 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet 6142f7c9286c44b58ebe162fd82b077f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:02 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:54068
I20260811 06:24:02.975203 16200 tablet_replica.cc:330] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: stopping tablet replica
I20260811 06:24:02.975474 16200 raft_consensus.cc:2229] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:02.976541 16200 raft_consensus.cc:2258] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.977758 16200 ts_tablet_manager.cc:1905] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.980783 16201 tablet_replica.cc:330] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: stopping tablet replica
I20260811 06:24:02.980997 16201 raft_consensus.cc:2229] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:02.981256 16201 raft_consensus.cc:2258] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.982445 16201 ts_tablet_manager.cc:1905] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.990963 16202 tablet_replica.cc:330] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: stopping tablet replica
I20260811 06:24:02.991372 16202 raft_consensus.cc:2229] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:24:02.992038 16202 raft_consensus.cc:2258] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:02.993583 16200 ts_tablet_manager.cc:1918] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:02.993739 16200 log.cc:1196] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/6142f7c9286c44b58ebe162fd82b077f
I20260811 06:24:02.996335 16201 ts_tablet_manager.cc:1918] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:02.997027 16200 ts_tablet_manager.cc:1939] T 6142f7c9286c44b58ebe162fd82b077f P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting consensus metadata
I20260811 06:24:02.997450 16202 ts_tablet_manager.cc:1905] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:02.998335 15543 catalog_manager.cc:4841] TS 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975): tablet 6142f7c9286c44b58ebe162fd82b077f (table test-table [id=2a6411ea5ceb452f8023fd41c7165980]) successfully deleted
I20260811 06:24:02.998708 16201 log.cc:1196] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/6142f7c9286c44b58ebe162fd82b077f
I20260811 06:24:02.999382 16201 ts_tablet_manager.cc:1939] T 6142f7c9286c44b58ebe162fd82b077f P cb139d9ca529467a9f551b7b051065a2: Deleting consensus metadata
I20260811 06:24:03.000514 15543 catalog_manager.cc:4841] TS cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947): tablet 6142f7c9286c44b58ebe162fd82b077f (table test-table [id=2a6411ea5ceb452f8023fd41c7165980]) successfully deleted
I20260811 06:24:03.009778 16202 ts_tablet_manager.cc:1918] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:03.010082 16202 log.cc:1196] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/6142f7c9286c44b58ebe162fd82b077f
I20260811 06:24:03.010648 16202 ts_tablet_manager.cc:1939] T 6142f7c9286c44b58ebe162fd82b077f P a7ce06edb00044f0a0d29b66ab73b503: Deleting consensus metadata
I20260811 06:24:03.011947 15546 catalog_manager.cc:4841] TS a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427): tablet 6142f7c9286c44b58ebe162fd82b077f (table test-table [id=2a6411ea5ceb452f8023fd41c7165980]) successfully deleted
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:24:03.068269 16167 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:03.055146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.7.4.190:45059)
0811 06:24:03.055390 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.055397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.055916 (+ 519us) server_negotiation.cc:407] Connection header received
0811 06:24:03.056178 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.056184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.056274 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.056379 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:03.058130 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.059456 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.061326 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.061978 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.062110 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.067966 (+ 5856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55188: BlockingRecv error: recv got EOF from 127.0.0.1:55188 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM
I20260811 06:24:03.092348 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.072053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.7.4.190:45059)
0811 06:24:03.072249 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.072255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.075015 (+ 2760us) server_negotiation.cc:407] Connection header received
0811 06:24:03.075273 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.075279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.075389 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.075483 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:03.077043 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.078138 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.080060 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.080709 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.080836 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.085618 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:03.085625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:03.085631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:03.085673 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:03.089209 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.090129 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.090135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.090138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.090312 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.090770 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.090775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.090779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.091112 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:03.091404 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.091728 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.091971 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:24:03.096412 15562 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:24:03.097143 15562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:03.110838 15708 tablet_service.cc:1467] Processing CreateTablet for tablet 96f3401b0d364ff191fea5183960bd42 (DEFAULT_TABLE table=test-table [id=8302c5d609c6420fbe58bc616f34d783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.111320 15708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f3401b0d364ff191fea5183960bd42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.115681 16018 tablet_service.cc:1467] Processing CreateTablet for tablet 96f3401b0d364ff191fea5183960bd42 (DEFAULT_TABLE table=test-table [id=8302c5d609c6420fbe58bc616f34d783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.116142 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f3401b0d364ff191fea5183960bd42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.123517 15865 tablet_service.cc:1467] Processing CreateTablet for tablet 96f3401b0d364ff191fea5183960bd42 (DEFAULT_TABLE table=test-table [id=8302c5d609c6420fbe58bc616f34d783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.123944 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96f3401b0d364ff191fea5183960bd42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.142836 16217 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Bootstrap starting.
I20260811 06:24:03.144655 16217 tablet_bootstrap.cc:654] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.146186 16216 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Bootstrap starting.
I20260811 06:24:03.148159 16216 tablet_bootstrap.cc:654] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.150578 16215 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Bootstrap starting.
I20260811 06:24:03.152488 16217 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: No bootstrap required, opened a new log
I20260811 06:24:03.152726 16215 tablet_bootstrap.cc:654] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.152994 16217 ts_tablet_manager.cc:1397] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 06:24:03.153802 16217 raft_consensus.cc:348] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.154066 16217 raft_consensus.cc:374] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.154163 16217 raft_consensus.cc:729] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.154414 16217 consensus_queue.cc:260] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.155014 16217 ts_tablet_manager.cc:1428] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:24:03.161139 16216 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: No bootstrap required, opened a new log
I20260811 06:24:03.161315 16216 ts_tablet_manager.cc:1397] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260811 06:24:03.162264 16216 raft_consensus.cc:348] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.162547 16216 raft_consensus.cc:374] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.162654 16216 raft_consensus.cc:729] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.162914 16216 consensus_queue.cc:260] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.163437 16215 tablet_bootstrap.cc:492] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: No bootstrap required, opened a new log
I20260811 06:24:03.163662 16215 ts_tablet_manager.cc:1397] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260811 06:24:03.164556 16215 raft_consensus.cc:348] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.164901 16215 raft_consensus.cc:374] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.165063 16215 raft_consensus.cc:729] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.165393 16215 consensus_queue.cc:260] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.165625 16216 ts_tablet_manager.cc:1428] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:24:03.168322 16215 ts_tablet_manager.cc:1428] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:24:03.186975 16144 raft_consensus.cc:482] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:03.187229 16144 raft_consensus.cc:504] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.188091 16144 leader_election.cc:288] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:03.188763 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f3401b0d364ff191fea5183960bd42" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" is_pre_election: true
I20260811 06:24:03.189038 15885 raft_consensus.cc:2454] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 0.
I20260811 06:24:03.188998 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f3401b0d364ff191fea5183960bd42" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503" is_pre_election: true
I20260811 06:24:03.189262 16040 raft_consensus.cc:2454] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 0.
I20260811 06:24:03.189570 15656 leader_election.cc:302] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:03.190001 16144 raft_consensus.cc:2781] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:03.190126 16144 raft_consensus.cc:482] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:03.190241 16144 raft_consensus.cc:3037] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.193286 16144 raft_consensus.cc:504] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.194389 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f3401b0d364ff191fea5183960bd42" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
I20260811 06:24:03.194442 16040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96f3401b0d364ff191fea5183960bd42" candidate_uuid: "cb139d9ca529467a9f551b7b051065a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ce06edb00044f0a0d29b66ab73b503"
I20260811 06:24:03.194607 15885 raft_consensus.cc:3037] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.194687 16040 raft_consensus.cc:3037] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.197917 15885 raft_consensus.cc:2454] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 1.
I20260811 06:24:03.197971 16040 raft_consensus.cc:2454] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb139d9ca529467a9f551b7b051065a2 in term 1.
I20260811 06:24:03.198487 15659 leader_election.cc:302] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:03.198794 16144 leader_election.cc:288] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [CANDIDATE]: Term 1 election: Requested vote from peers 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975), a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427)
I20260811 06:24:03.199033 16144 raft_consensus.cc:2781] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:03.199326 16144 raft_consensus.cc:686] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 1 LEADER]: Becoming Leader. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Running, Role: LEADER
I20260811 06:24:03.199671 16144 consensus_queue.cc:237] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.211923 15560 catalog_manager.cc:5495] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb139d9ca529467a9f551b7b051065a2 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "cb139d9ca529467a9f551b7b051065a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } health_report { overall_health: UNKNOWN } } }
I20260811 06:24:03.252063 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.241225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.7.4.129:42947)
0811 06:24:03.245325 (+ 4100us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.245331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.245358 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:24:03.245433 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.245438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.245528 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.245646 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:03.246135 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.247389 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.249535 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.250256 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.250403 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.250910 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:03.251054 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:03.251170 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.251359 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.251446 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3934,"thread_start_us":3777,"threads_started":1}
I20260811 06:24:03.255510 16040 raft_consensus.cc:1261] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Refusing update from remote peer cb139d9ca529467a9f551b7b051065a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:03.255683 15885 raft_consensus.cc:1261] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Refusing update from remote peer cb139d9ca529467a9f551b7b051065a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:03.256363 16144 consensus_queue.cc:1035] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:03.257035 16144 consensus_queue.cc:1035] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:03.281577 15562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:03.281852 15562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:03.285873 15562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Sending DeleteTablet for 3 replicas of tablet 96f3401b0d364ff191fea5183960bd42
I20260811 06:24:03.287492 15708 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f3401b0d364ff191fea5183960bd42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:41720
I20260811 06:24:03.287791 16201 tablet_replica.cc:330] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: stopping tablet replica
I20260811 06:24:03.288021 16201 raft_consensus.cc:2229] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:24:03.288394 16201 raft_consensus.cc:2258] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.288517 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f3401b0d364ff191fea5183960bd42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:33976
I20260811 06:24:03.288869 16200 tablet_replica.cc:330] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: stopping tablet replica
I20260811 06:24:03.289083 16200 raft_consensus.cc:2229] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:03.289354 16200 raft_consensus.cc:2258] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.290068 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet 96f3401b0d364ff191fea5183960bd42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:54068
I20260811 06:24:03.290609 15560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:24:03.291231 15560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:03.291469 16200 ts_tablet_manager.cc:1905] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.296347 16201 ts_tablet_manager.cc:1905] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.296482 16202 tablet_replica.cc:330] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: stopping tablet replica
I20260811 06:24:03.296715 16202 raft_consensus.cc:2229] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:03.296892 16202 pending_rounds.cc:62] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Trying to abort 2 pending ops.
I20260811 06:24:03.297017 16202 pending_rounds.cc:68] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317214999566880768 op_type: WRITE_OP write_request { tablet_id: "96f3401b0d364ff191fea5183960bd42" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\257\202\353\323\006\"8\n\ntest-admin\022*\n 8302c5d609c6420fbe58bc616f34d783\020\001\030\001 \001(\001" signature: "<redacted>""\023\340A\365\212p\300\377\365\271\351\346\233\325T\270\236Y?%\317\204\202\334\321\007\205\tuoc\3563^\177\307\343\036*\000\227\244;\307\000T%\201\270z#\241\305%\2576\200\370K\261T\310\014\034" signing_key_seq_num: 0 } } request_id { client_id: "31be6d054ed0430885877f909438507a" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260811 06:24:03.297834 16202 pending_rounds.cc:68] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317214999348822016 op_type: NO_OP noop_request { }
I20260811 06:24:03.298009 16202 raft_consensus.cc:2866] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 06:24:03.298143 16202 raft_consensus.cc:2258] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.300420 16202 ts_tablet_manager.cc:1905] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.303068 16201 ts_tablet_manager.cc:1918] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:03.303225 16201 log.cc:1196] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/96f3401b0d364ff191fea5183960bd42
I20260811 06:24:03.306818 16018 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c (DEFAULT_TABLE table=test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.307273 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9ae65dd7784c79b6dc25a065a1e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.309463 16200 ts_tablet_manager.cc:1918] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:03.309660 16200 log.cc:1196] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/96f3401b0d364ff191fea5183960bd42
I20260811 06:24:03.310103 16200 ts_tablet_manager.cc:1939] T 96f3401b0d364ff191fea5183960bd42 P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting consensus metadata
I20260811 06:24:03.311587 16201 ts_tablet_manager.cc:1939] T 96f3401b0d364ff191fea5183960bd42 P cb139d9ca529467a9f551b7b051065a2: Deleting consensus metadata
I20260811 06:24:03.311650 15543 catalog_manager.cc:4841] TS 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975): tablet 96f3401b0d364ff191fea5183960bd42 (table test-table [id=8302c5d609c6420fbe58bc616f34d783]) successfully deleted
I20260811 06:24:03.312425 15708 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c (DEFAULT_TABLE table=test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.312712 15543 catalog_manager.cc:4841] TS cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947): tablet 96f3401b0d364ff191fea5183960bd42 (table test-table [id=8302c5d609c6420fbe58bc616f34d783]) successfully deleted
I20260811 06:24:03.312845 15708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9ae65dd7784c79b6dc25a065a1e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.315897 15865 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c (DEFAULT_TABLE table=test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:03.316340 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f9ae65dd7784c79b6dc25a065a1e50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:03.336769 16217 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Bootstrap starting.
I20260811 06:24:03.338599 16217 tablet_bootstrap.cc:654] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.339702 16215 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Bootstrap starting.
I20260811 06:24:03.341148 16202 ts_tablet_manager.cc:1918] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:03.341683 16216 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Bootstrap starting.
I20260811 06:24:03.341938 16202 log.cc:1196] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/96f3401b0d364ff191fea5183960bd42
I20260811 06:24:03.342495 16202 ts_tablet_manager.cc:1939] T 96f3401b0d364ff191fea5183960bd42 P a7ce06edb00044f0a0d29b66ab73b503: Deleting consensus metadata
I20260811 06:24:03.343277 16216 tablet_bootstrap.cc:654] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.341459 16215 tablet_bootstrap.cc:654] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:03.345381 15546 catalog_manager.cc:4841] TS a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427): tablet 96f3401b0d364ff191fea5183960bd42 (table test-table [id=8302c5d609c6420fbe58bc616f34d783]) successfully deleted
I20260811 06:24:03.354869 16215 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: No bootstrap required, opened a new log
I20260811 06:24:03.355111 16215 ts_tablet_manager.cc:1397] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260811 06:24:03.356280 16215 raft_consensus.cc:348] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.356665 16215 raft_consensus.cc:374] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.356778 16215 raft_consensus.cc:729] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.357131 16215 consensus_queue.cc:260] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.357749 16215 ts_tablet_manager.cc:1428] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:03.358273 16217 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: No bootstrap required, opened a new log
I20260811 06:24:03.358522 16217 ts_tablet_manager.cc:1397] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.004s
I20260811 06:24:03.359473 16217 raft_consensus.cc:348] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.359822 16217 raft_consensus.cc:374] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.359926 16217 raft_consensus.cc:729] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb139d9ca529467a9f551b7b051065a2, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.360191 16217 consensus_queue.cc:260] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.360918 16217 ts_tablet_manager.cc:1428] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:03.364179 16216 tablet_bootstrap.cc:492] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: No bootstrap required, opened a new log
I20260811 06:24:03.364365 16216 ts_tablet_manager.cc:1397] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.004s
I20260811 06:24:03.365098 16216 raft_consensus.cc:348] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.365337 16216 raft_consensus.cc:374] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:03.365442 16216 raft_consensus.cc:729] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa65b76c54cbf9bb1d8ecbb4d96d1, State: Initialized, Role: FOLLOWER
I20260811 06:24:03.365715 16216 consensus_queue.cc:260] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.366246 16216 ts_tablet_manager.cc:1428] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:24:03.401328 16157 raft_consensus.cc:482] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:03.401574 16157 raft_consensus.cc:504] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.402208 16157 leader_election.cc:288] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975)
I20260811 06:24:03.402846 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9ae65dd7784c79b6dc25a065a1e50c" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2" is_pre_election: true
I20260811 06:24:03.402992 15887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9ae65dd7784c79b6dc25a065a1e50c" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" is_pre_election: true
I20260811 06:24:03.403129 15726 raft_consensus.cc:2454] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 0.
I20260811 06:24:03.403223 15887 raft_consensus.cc:2454] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 0.
I20260811 06:24:03.403649 15963 leader_election.cc:302] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7ce06edb00044f0a0d29b66ab73b503, cb139d9ca529467a9f551b7b051065a2; no voters:
I20260811 06:24:03.403987 16157 raft_consensus.cc:2781] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:03.404098 16157 raft_consensus.cc:482] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:03.404245 16157 raft_consensus.cc:3037] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.406798 16157 raft_consensus.cc:504] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.407426 16157 leader_election.cc:288] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 election: Requested vote from peers cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947), 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975)
I20260811 06:24:03.407923 15726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9ae65dd7784c79b6dc25a065a1e50c" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb139d9ca529467a9f551b7b051065a2"
I20260811 06:24:03.408010 15887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9ae65dd7784c79b6dc25a065a1e50c" candidate_uuid: "a7ce06edb00044f0a0d29b66ab73b503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1"
I20260811 06:24:03.408154 15726 raft_consensus.cc:3037] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.408217 15887 raft_consensus.cc:3037] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:03.410207 15887 raft_consensus.cc:2454] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 1.
I20260811 06:24:03.410616 15963 leader_election.cc:302] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492fa65b76c54cbf9bb1d8ecbb4d96d1, a7ce06edb00044f0a0d29b66ab73b503; no voters:
I20260811 06:24:03.410869 16157 raft_consensus.cc:2781] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:03.411100 16157 raft_consensus.cc:686] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 1 LEADER]: Becoming Leader. State: Replica: a7ce06edb00044f0a0d29b66ab73b503, State: Running, Role: LEADER
I20260811 06:24:03.411314 15726 raft_consensus.cc:2454] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ce06edb00044f0a0d29b66ab73b503 in term 1.
I20260811 06:24:03.411415 16157 consensus_queue.cc:237] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } }
I20260811 06:24:03.414711 15562 catalog_manager.cc:5495] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131). New cstate: current_term: 1 leader_uuid: "a7ce06edb00044f0a0d29b66ab73b503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7ce06edb00044f0a0d29b66ab73b503" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 46427 } health_report { overall_health: HEALTHY } } }
I20260811 06:24:03.452160 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.446272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.7.4.131:46427)
0811 06:24:03.446874 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.446878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.446904 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:24:03.446951 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.446954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.447009 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.447115 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:03.447565 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.448678 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.450185 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.450662 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.450825 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.451356 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:03.451481 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:03.451589 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.451753 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.451832 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":292,"threads_started":1}
I20260811 06:24:03.462038 15887 raft_consensus.cc:1261] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Refusing update from remote peer a7ce06edb00044f0a0d29b66ab73b503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:03.462085 15726 raft_consensus.cc:1261] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Refusing update from remote peer a7ce06edb00044f0a0d29b66ab73b503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:03.462594 16198 consensus_queue.cc:1035] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb139d9ca529467a9f551b7b051065a2" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:03.463085 16157 consensus_queue.cc:1035] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492fa65b76c54cbf9bb1d8ecbb4d96d1" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:03.515450 15562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:03.515718 15562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:03.518500 15562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 077242adbd244cbeb8404a792a04e12f: Sending DeleteTablet for 3 replicas of tablet 7f9ae65dd7784c79b6dc25a065a1e50c
I20260811 06:24:03.519272 15708 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:41720
I20260811 06:24:03.519659 16201 tablet_replica.cc:330] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: stopping tablet replica
I20260811 06:24:03.519654 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:54068
I20260811 06:24:03.519851 16201 raft_consensus.cc:2229] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:03.519970 16202 tablet_replica.cc:330] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: stopping tablet replica
I20260811 06:24:03.519958 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9ae65dd7784c79b6dc25a065a1e50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:03 UTC) from {username='oryx', principal='oryx/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:33976
I20260811 06:24:03.520072 16201 raft_consensus.cc:2258] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.520179 16202 raft_consensus.cc:2229] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:24:03.520349 16200 tablet_replica.cc:330] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: stopping tablet replica
I20260811 06:24:03.520493 16202 raft_consensus.cc:2258] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.520568 16200 raft_consensus.cc:2229] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:03.520753 16200 raft_consensus.cc:2258] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:03.521271 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15639
I20260811 06:24:03.521631 16202 ts_tablet_manager.cc:1905] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.522863 16201 ts_tablet_manager.cc:1905] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.526546 16202 ts_tablet_manager.cc:1918] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:03.526638 16202 log.cc:1196] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/7f9ae65dd7784c79b6dc25a065a1e50c
I20260811 06:24:03.526942 16202 ts_tablet_manager.cc:1939] T 7f9ae65dd7784c79b6dc25a065a1e50c P a7ce06edb00044f0a0d29b66ab73b503: Deleting consensus metadata
I20260811 06:24:03.527998 15546 catalog_manager.cc:4841] TS a7ce06edb00044f0a0d29b66ab73b503 (127.7.4.131:46427): tablet 7f9ae65dd7784c79b6dc25a065a1e50c (table test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]) successfully deleted
I20260811 06:24:03.528242 16200 ts_tablet_manager.cc:1905] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:03.531934 16201 ts_tablet_manager.cc:1918] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:03.532140 16201 log.cc:1196] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/7f9ae65dd7784c79b6dc25a065a1e50c
I20260811 06:24:03.532575 16201 ts_tablet_manager.cc:1939] T 7f9ae65dd7784c79b6dc25a065a1e50c P cb139d9ca529467a9f551b7b051065a2: Deleting consensus metadata
I20260811 06:24:03.533116 16200 ts_tablet_manager.cc:1918] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:24:03.533258 16200 log.cc:1196] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/7f9ae65dd7784c79b6dc25a065a1e50c
I20260811 06:24:03.533764 16200 ts_tablet_manager.cc:1939] T 7f9ae65dd7784c79b6dc25a065a1e50c P 492fa65b76c54cbf9bb1d8ecbb4d96d1: Deleting consensus metadata
I20260811 06:24:03.533710 15543 catalog_manager.cc:4841] TS cb139d9ca529467a9f551b7b051065a2 (127.7.4.129:42947): tablet 7f9ae65dd7784c79b6dc25a065a1e50c (table test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]) successfully deleted
I20260811 06:24:03.534631 15543 catalog_manager.cc:4841] TS 492fa65b76c54cbf9bb1d8ecbb4d96d1 (127.7.4.130:33975): tablet 7f9ae65dd7784c79b6dc25a065a1e50c (table test-table [id=64c52e4976e94bd28acc4ce7ddbd1beb]) successfully deleted
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.129@KRBTEST.COM
I20260811 06:24:03.546533 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.522862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.7.4.129:42947)
0811 06:24:03.523052 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.523058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.532674 (+ 9616us) server_negotiation.cc:407] Connection header received
0811 06:24:03.532895 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.532901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.532987 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.533114 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:03.534553 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.535392 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.536581 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.537000 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.537102 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.540909 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:03.540925 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:03.540941 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:03.540984 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:03.544017 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.544593 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.544600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.544602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.544728 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.545057 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.545060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.545062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.545472 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:03.545711 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.545925 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.546109 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:24:03.547183 15778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:03.687870 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15639
I20260811 06:24:03.716724 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15792
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.130@KRBTEST.COM
I20260811 06:24:03.731969 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.717495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.7.4.130:33975)
0811 06:24:03.717985 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.717989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.718006 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:24:03.718062 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.718066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.718127 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.718218 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:03.719550 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.720376 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.721588 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.721990 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.722081 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.726016 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:03.726030 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:03.726044 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:03.726075 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:03.728998 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.729781 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.729796 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.729799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.729943 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.730351 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.730357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.730360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.730714 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:03.731036 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.731344 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.731600 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
I20260811 06:24:03.732616 15935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:03.866686 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15792
I20260811 06:24:03.893608 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15946
Aug 11 06:24:03 dist-test-slave-07c2 krb5kdc[15514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.4.131@KRBTEST.COM
I20260811 06:24:03.909399 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:03.894363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54088 (local address 127.7.4.131:46427)
0811 06:24:03.894546 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:03.894554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:03.894679 (+ 125us) server_negotiation.cc:407] Connection header received
0811 06:24:03.894896 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:03.894901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:03.894967 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:03.895088 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:03.896501 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.897282 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.898566 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:03.899001 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:03.899143 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:03.903553 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:03.903568 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:03.903584 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:03.903618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:03.906521 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.907208 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.907220 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.907228 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.907434 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:03.907825 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:03.907830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:03.907834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:03.908200 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:03.908522 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:03.908818 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 06:24:03.909046 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:24:03.910053 16086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:04.036441 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15946
I20260811 06:24:04.064292 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 15527
I20260811 06:24:04.065550 15606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:04.183634 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 15527
2026-08-11T06:24:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5954 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:24:04 dist-test-slave-07c2 krb5kdc[16249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:24:04 dist-test-slave-07c2 krb5kdc[16249](info): set up 2 sockets
Aug 11 06:24:04 dist-test-slave-07c2 krb5kdc[16249](info): commencing operation
krb5kdc: starting...
W20260811 06:24:05.344852 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:24:05 dist-test-slave-07c2 krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:24:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:24:05.599282 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34639
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:05.923622 16263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:05.923878 16263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:05.923982 16263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:05.930787 16263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:05.930876 16263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:05.930924 16263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:05.930976 16263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:05.931015 16263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:05.940164 16263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:05.941207 16263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:05.942416 16263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:05.953655 16273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:05.953737 16270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:05.953702 16271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:05.954480 16263 server_base.cc:1034] running on GCE node
I20260811 06:24:05.955246 16263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:05.956624 16263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:05.957854 16263 hybrid_clock.cc:648] HybridClock initialized: now 1786429445957813 us; error 45 us; skew 500 ppm
I20260811 06:24:05.958215 16263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T06:24:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1840 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:24:06 dist-test-slave-07c2 krb5kdc[16279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:24:06 dist-test-slave-07c2 krb5kdc[16279](info): set up 2 sockets
Aug 11 06:24:06 dist-test-slave-07c2 krb5kdc[16279](info): commencing operation
krb5kdc: starting...
W20260811 06:24:07.187759 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:24:07 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:24:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:24:07.462317 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34087
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:07.794090 16294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:07.794359 16294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:07.794515 16294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:07.801517 16294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:07.801613 16294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:07.801657 16294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:07.801745 16294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:07.801831 16294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:07.811389 16294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:07.812661 16294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:07.813917 16294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:07.825604 16301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:07.825551 16303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:07.826119 16300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:07.826622 16294 server_base.cc:1034] running on GCE node
I20260811 06:24:07.827454 16294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:07.828748 16294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:07.829954 16294 hybrid_clock.cc:648] HybridClock initialized: now 1786429447829898 us; error 38 us; skew 500 ppm
I20260811 06:24:07.830245 16294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:07 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:07.836757 16294 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:24:07.838100 16294 webserver.cc:460] Webserver started at http://127.7.4.190:41309/ using document root <none> and password file <none>
I20260811 06:24:07.838706 16294 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:07.838809 16294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:07.839072 16294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:07.841950 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "7135704df62b4332bd9ea2ca52c8a162"
format_stamp: "Formatted at 2026-08-11 06:24:07 on dist-test-slave-07c2"
server_key: "ae59a8e9b5d76231417a997f0211c7a0"
server_key_iv: "a5db009d4053e467c5a740cf55bfccbf"
server_key_version: "encryptionkey@0"
I20260811 06:24:07.842691 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "7135704df62b4332bd9ea2ca52c8a162"
format_stamp: "Formatted at 2026-08-11 06:24:07 on dist-test-slave-07c2"
server_key: "ae59a8e9b5d76231417a997f0211c7a0"
server_key_iv: "a5db009d4053e467c5a740cf55bfccbf"
server_key_version: "encryptionkey@0"
I20260811 06:24:07.847694 16294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:24:07.850847 16310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:07.852154 16294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:07.852293 16294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "7135704df62b4332bd9ea2ca52c8a162"
format_stamp: "Formatted at 2026-08-11 06:24:07 on dist-test-slave-07c2"
server_key: "ae59a8e9b5d76231417a997f0211c7a0"
server_key_iv: "a5db009d4053e467c5a740cf55bfccbf"
server_key_version: "encryptionkey@0"
I20260811 06:24:07.852445 16294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:07.866037 16294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:07.869755 16294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:07.869967 16294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:07.888622 16294 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:34087
I20260811 06:24:07.888659 16364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:34087 every 8 connection(s)
I20260811 06:24:07.889922 16294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:07.893380 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:07.897203 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16294
I20260811 06:24:07.897451 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:24:07.898270 7186 external_mini_cluster.cc:1428] Setting key 847382c39ffd481b6b50b355283bed8a
I20260811 06:24:07.904469 16365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Bootstrap starting.
I20260811 06:24:07.907390 16365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:07.908358 16365 log.cc:824] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:07.911007 16365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: No bootstrap required, opened a new log
Aug 11 06:24:07 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:07.919831 16365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:07.920204 16365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:07.920336 16365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135704df62b4332bd9ea2ca52c8a162, State: Initialized, Role: FOLLOWER
I20260811 06:24:07.920823 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:07.900307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.7.4.190:34087)
0811 06:24:07.901032 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:07.901043 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:07.901081 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:24:07.903280 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:07.903359 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:07.903708 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:07.904081 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:07.905582 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:07.906837 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:07.908152 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:07.909032 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:07.909235 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:07.913481 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:07.913488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:07.913500 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:07.913535 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:07.916628 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:07.917452 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:07.917458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:07.917463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:07.917634 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:07.918060 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:07.918063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:07.918065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:07.918550 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:07.918783 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:07.919222 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 06:24:07.919664 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":345,"threads_started":1}
I20260811 06:24:07.920912 16365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:07.921242 16365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:07.921413 16365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:07.921617 16365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:07.924640 16365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:07.925199 16365 leader_election.cc:302] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7135704df62b4332bd9ea2ca52c8a162; no voters:
I20260811 06:24:07.925792 16365 leader_election.cc:288] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:24:07.925932 16371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:07.927366 16371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 LEADER]: Becoming Leader. State: Replica: 7135704df62b4332bd9ea2ca52c8a162, State: Running, Role: LEADER
I20260811 06:24:07.928001 16365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:07.927870 16371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:07.931126 16372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7135704df62b4332bd9ea2ca52c8a162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } } }
I20260811 06:24:07.931203 16373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7135704df62b4332bd9ea2ca52c8a162. Latest consensus state: current_term: 1 leader_uuid: "7135704df62b4332bd9ea2ca52c8a162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } } }
I20260811 06:24:07.931576 16372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:07.931619 16373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:07.933329 16381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:07.939028 16381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:07.947692 16381 catalog_manager.cc:1347] Generated new cluster ID: a1294c6c8bdb4c5384b544f9cecd4eb5
I20260811 06:24:07.947851 16381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:07.961339 16381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:24:07.962549 16381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:07.972050 16381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Generated new TSK 0
I20260811 06:24:07.972941 16381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:24:08.048345 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:08.402726 16395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:08.403002 16395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:08.403100 16395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:08.410830 16395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:08.410945 16395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:08.411147 16395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:08.420930 16395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:08.422255 16395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:08.423691 16395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:08.437569 16402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:08.437837 16404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:08.437834 16401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:08.438828 16395 server_base.cc:1034] running on GCE node
I20260811 06:24:08.439570 16395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:08.440778 16395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:08.442060 16395 hybrid_clock.cc:648] HybridClock initialized: now 1786429448441945 us; error 97 us; skew 500 ppm
I20260811 06:24:08.442482 16395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:08 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429448, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:08.445784 16395 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:24:08.447259 16395 webserver.cc:460] Webserver started at http://127.7.4.129:46633/ using document root <none> and password file <none>
I20260811 06:24:08.447988 16395 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:08.448153 16395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:08.448453 16395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:08.451427 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "4601a32d6e20466b940c2d9f1f84b09e"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "b1f146c1bc96304f5724f2c4694a757a"
server_key_iv: "6ce6936f39832a0ef3a98e6b7873f8d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.452189 16395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "4601a32d6e20466b940c2d9f1f84b09e"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "b1f146c1bc96304f5724f2c4694a757a"
server_key_iv: "6ce6936f39832a0ef3a98e6b7873f8d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.457525 16395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:24:08.461054 16411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:08.462451 16395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:24:08.462617 16395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "4601a32d6e20466b940c2d9f1f84b09e"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "b1f146c1bc96304f5724f2c4694a757a"
server_key_iv: "6ce6936f39832a0ef3a98e6b7873f8d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.462800 16395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:08.488404 16395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:08.492339 16395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:08.492643 16395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:08.493582 16395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:08.494872 16395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:08.494946 16395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:08.495103 16395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:08.495162 16395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:08.530050 16395 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:36539
I20260811 06:24:08.530138 16536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:36539 every 8 connection(s)
I20260811 06:24:08.531370 16395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:08.539906 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16395
I20260811 06:24:08.540117 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:24:08.540850 7186 external_mini_cluster.cc:1428] Setting key 9bdb6ceb96bc1a657d0ed8ee43605f50
Aug 11 06:24:08 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429448, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:08.553406 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:08.534175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:38979 (local address 127.7.4.190:34087)
0811 06:24:08.534623 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:08.534628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:08.535649 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 06:24:08.537508 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:08.537514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:08.537607 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:08.537730 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:08.539210 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:08.540426 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:08.542374 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:08.542793 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:08.542927 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:08.547410 (+ 4483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:08.547415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:08.547419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:08.547449 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:08.550080 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:08.551024 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:08.551027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:08.551030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:08.551146 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:08.551676 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:08.551680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:08.551682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:08.551950 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:08.552073 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:08.552968 (+ 895us) server_negotiation.cc:299] Negotiation successful
0811 06:24:08.553159 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260811 06:24:08.554867 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:08.534747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:38979)
0811 06:24:08.535490 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:08.535536 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:08.537225 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:08.537963 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:08.537971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:08.538471 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:08.539025 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:08.539043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:08.540620 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:08.540624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:08.542232 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:08.542255 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:08.543018 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:08.543023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:08.543220 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:08.543956 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:08.543977 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:08.547131 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:08.550245 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:08.550257 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:08.550274 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:08.550907 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:08.551259 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:08.551267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:08.551274 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:08.551549 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:08.552085 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:08.552095 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:08.552546 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 06:24:08.552849 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 06:24:08.553419 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":279,"threads_started":1}
I20260811 06:24:08.556695 16537 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34087
I20260811 06:24:08.557013 16537 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:08.557777 16537 heartbeater.cc:511] Master 127.7.4.190:34087 requested a full tablet report, sending...
I20260811 06:24:08.560060 16328 ts_manager.cc:194] Registered new tserver with Master: 4601a32d6e20466b940c2d9f1f84b09e (127.7.4.129:36539)
I20260811 06:24:08.562227 16328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:38979
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:24:08.602103 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:08.930508 16545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:08.930795 16545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:08.930950 16545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:08.938068 16545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:08.938218 16545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:08.938383 16545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:08.947495 16545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:08.948684 16545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:08.949873 16545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:08.963701 16553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:08.963657 16550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:08.963753 16551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:08.968602 16545 server_base.cc:1034] running on GCE node
I20260811 06:24:08.969201 16545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:08.970193 16545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:08.971387 16545 hybrid_clock.cc:648] HybridClock initialized: now 1786429448971335 us; error 43 us; skew 500 ppm
I20260811 06:24:08.971683 16545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:08 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429448, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:08.974763 16545 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:24:08.976222 16545 webserver.cc:460] Webserver started at http://127.7.4.130:37339/ using document root <none> and password file <none>
I20260811 06:24:08.976910 16545 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:08.977061 16545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:08.977373 16545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:08.980269 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "8fc58aaebcc2484f909a5051c962cea6"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "d9573613f9e728f6a8508874a9e9b786"
server_key_iv: "b353a36368316183a39f537f7cce0436"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.981009 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "8fc58aaebcc2484f909a5051c962cea6"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "d9573613f9e728f6a8508874a9e9b786"
server_key_iv: "b353a36368316183a39f537f7cce0436"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.985963 16545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:24:08.989238 16560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:08.990432 16545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:08.990598 16545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "8fc58aaebcc2484f909a5051c962cea6"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "d9573613f9e728f6a8508874a9e9b786"
server_key_iv: "b353a36368316183a39f537f7cce0436"
server_key_version: "encryptionkey@0"
I20260811 06:24:08.990814 16545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:09.027336 16545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:09.031702 16545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:09.032006 16545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:09.033198 16545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:09.034533 16545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:09.034654 16545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:09.034795 16545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:09.034866 16545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:09.068851 16545 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:44359
I20260811 06:24:09.068938 16682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:44359 every 8 connection(s)
I20260811 06:24:09.070109 16545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:09.076226 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16545
I20260811 06:24:09.076462 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:24:09.076942 7186 external_mini_cluster.cc:1428] Setting key f37d1c39d3cd02dc827aa25e83c39dac
Aug 11 06:24:09 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429448, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:09.092914 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.072738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:35823 (local address 127.7.4.190:34087)
0811 06:24:09.073192 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:09.073198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:09.074168 (+ 970us) server_negotiation.cc:407] Connection header received
0811 06:24:09.076032 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:09.076039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:09.076130 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:09.076270 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:09.077859 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.078928 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.081361 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.081774 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.081868 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.086066 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:09.086075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:09.086081 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:09.086124 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:09.089788 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.090478 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.090485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.090492 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.090661 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.091091 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.091098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.091104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.091494 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:09.091666 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:09.092485 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 06:24:09.092641 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":204,"threads_started":1}
I20260811 06:24:09.094409 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.073326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:35823)
0811 06:24:09.074014 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:09.074052 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:09.075747 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:09.076558 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:09.076575 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:09.077125 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:09.077667 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:09.077697 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.079122 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.079127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:09.081211 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:09.081229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.081996 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.082005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:09.082323 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.083079 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:09.083113 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:09.085830 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:09.089921 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:09.089927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:09.089937 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:09.090343 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:09.090781 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:09.090784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:09.090787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:09.090965 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:09.091640 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:09.091651 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:09.092085 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 06:24:09.092393 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 06:24:09.092951 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":287,"threads_started":1}
I20260811 06:24:09.095836 16684 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34087
I20260811 06:24:09.096153 16684 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:09.096860 16684 heartbeater.cc:511] Master 127.7.4.190:34087 requested a full tablet report, sending...
I20260811 06:24:09.098250 16328 ts_manager.cc:194] Registered new tserver with Master: 8fc58aaebcc2484f909a5051c962cea6 (127.7.4.130:44359)
I20260811 06:24:09.099035 16328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:35823
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:24:09.136554 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:09.464805 16690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:09.465041 16690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:09.465211 16690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:09.472476 16690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:09.472589 16690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:09.472780 16690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:09.481983 16690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:09.483189 16690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:09.484524 16690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:09.497403 16695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:09.497426 16698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:09.497371 16696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:09.498167 16690 server_base.cc:1034] running on GCE node
I20260811 06:24:09.498823 16690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:09.499809 16690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:09.501005 16690 hybrid_clock.cc:648] HybridClock initialized: now 1786429449500971 us; error 30 us; skew 500 ppm
I20260811 06:24:09.501309 16690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:09 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429449, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:09.504379 16690 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:24:09.505810 16690 webserver.cc:460] Webserver started at http://127.7.4.131:34525/ using document root <none> and password file <none>
I20260811 06:24:09.506485 16690 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:09.506633 16690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:09.506971 16690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:09.509938 16690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "c3563bcd2b0842dc80e7bdb76aa3134e"
format_stamp: "Formatted at 2026-08-11 06:24:09 on dist-test-slave-07c2"
server_key: "ad53c65f70b356e879cee144c2f93de9"
server_key_iv: "38493071efbc1927c95e508b186a8b7f"
server_key_version: "encryptionkey@0"
I20260811 06:24:09.510789 16690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "c3563bcd2b0842dc80e7bdb76aa3134e"
format_stamp: "Formatted at 2026-08-11 06:24:09 on dist-test-slave-07c2"
server_key: "ad53c65f70b356e879cee144c2f93de9"
server_key_iv: "38493071efbc1927c95e508b186a8b7f"
server_key_version: "encryptionkey@0"
I20260811 06:24:09.515830 16690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:24:09.519086 16705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:09.520323 16690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:09.520488 16690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "c3563bcd2b0842dc80e7bdb76aa3134e"
format_stamp: "Formatted at 2026-08-11 06:24:09 on dist-test-slave-07c2"
server_key: "ad53c65f70b356e879cee144c2f93de9"
server_key_iv: "38493071efbc1927c95e508b186a8b7f"
server_key_version: "encryptionkey@0"
I20260811 06:24:09.520685 16690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:09.546676 16690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:09.550714 16690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:09.551009 16690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:09.552314 16690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:09.553706 16690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:09.553817 16690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:09.553969 16690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:09.554042 16690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:09.565214 16537 heartbeater.cc:503] Master 127.7.4.190:34087 was elected leader, sending a full tablet report...
I20260811 06:24:09.592694 16690 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46403
I20260811 06:24:09.592741 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46403 every 8 connection(s)
I20260811 06:24:09.594067 16690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:09.600633 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16690
I20260811 06:24:09.600831 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:24:09.601256 7186 external_mini_cluster.cc:1428] Setting key 8779ec755a997cc253e4cb6ee8d317c3
Aug 11 06:24:09 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429449, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:09.613469 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.596805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:56637 (local address 127.7.4.190:34087)
0811 06:24:09.597231 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:09.597236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:09.598142 (+ 906us) server_negotiation.cc:407] Connection header received
0811 06:24:09.599963 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:09.599967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:09.600044 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:09.600142 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:09.601688 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.602578 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.604360 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.604753 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.604848 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.608033 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:09.608037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:09.608040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:09.608069 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:09.610514 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.611230 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.611234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.611236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.611401 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.611822 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.611827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.611830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.612072 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:09.612247 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:09.612858 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 06:24:09.613086 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260811 06:24:09.614292 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.597317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:56637)
0811 06:24:09.597979 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:09.598021 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:09.599729 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:09.600340 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:09.600347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:09.600895 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:09.601501 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:09.601517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.602749 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.602752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:09.604208 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:09.604238 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.604928 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.604933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:09.605107 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.605685 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:09.605705 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:09.607830 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:09.610655 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:09.610662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:09.610675 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:09.611090 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:09.611511 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:09.611516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:09.611519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:09.611694 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:09.612203 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:09.612210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:09.612504 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 06:24:09.612753 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 06:24:09.613236 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":269,"threads_started":1}
I20260811 06:24:09.615764 16830 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34087
I20260811 06:24:09.616073 16830 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:09.616803 16830 heartbeater.cc:511] Master 127.7.4.190:34087 requested a full tablet report, sending...
I20260811 06:24:09.618160 16327 ts_manager.cc:194] Registered new tserver with Master: c3563bcd2b0842dc80e7bdb76aa3134e (127.7.4.131:46403)
I20260811 06:24:09.618986 16327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:56637
I20260811 06:24:09.628806 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
Entry for principal oryx/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab.
WARNING: no policy specified for oryx/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
Entry for principal oryx/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab.
WARNING: no policy specified for oryx/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
Entry for principal oryx/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab.
I20260811 06:24:09.775532 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16395
Aug 11 06:24:09 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:24:09.791690 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.776680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.7.4.129:36539)
0811 06:24:09.777379 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:09.777386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:09.777412 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:24:09.777474 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:09.777480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:09.777699 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:09.777846 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:09.779150 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.780210 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.781399 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.781945 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.782040 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.785954 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:09.785967 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:09.785980 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:09.786019 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:09.788863 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.789551 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.789567 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.789570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.789703 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.790072 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.790078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.790081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.790449 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:09.790763 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:09.791055 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 06:24:09.791354 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":345,"threads_started":1}
I20260811 06:24:09.792963 16530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:09.875402 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16395
I20260811 06:24:09.896574 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16545
Aug 11 06:24:09 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:24:09.912415 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:09.897493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.7.4.130:44359)
0811 06:24:09.898036 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:09.898043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:09.898067 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:24:09.898124 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:09.898131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:09.898333 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:09.898500 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:09.899849 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.900841 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.902067 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:09.902592 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:09.902735 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:09.906930 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:09.906940 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:09.906952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:09.906990 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:09.909640 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.910331 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.910338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.910340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.910463 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:09.910813 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:09.910819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:09.910822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:09.911141 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:09.911502 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:09.911815 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:24:09.912075 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":262,"threads_started":1}
I20260811 06:24:09.913630 16677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:09.996395 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16545
I20260811 06:24:10.016412 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16690
Aug 11 06:24:10 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:24:10.033612 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:10.017449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.7.4.131:46403)
0811 06:24:10.018050 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:10.018058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:10.018087 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:24:10.018145 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:10.018152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:10.018383 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:10.018576 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:10.019989 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:10.020994 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:10.022330 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:10.022879 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:10.023028 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:10.027799 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:10.027811 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:10.027824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:10.027860 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:10.030807 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:10.031612 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:10.031619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:10.031622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:10.031756 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:10.032149 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:10.032152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:10.032154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:10.032482 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:10.032761 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:10.033011 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 06:24:10.033205 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":291,"threads_started":1}
I20260811 06:24:10.034888 16825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:10.117425 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16690
I20260811 06:24:10.138875 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16294
I20260811 06:24:10.140118 16360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:10.244270 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16294
I20260811 06:24:10.269501 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34087
--webserver_interface=127.7.4.190
--webserver_port=41309
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:10.617429 16852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:10.617786 16852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:10.617951 16852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:10.625522 16852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:10.625677 16852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:10.625789 16852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:10.625887 16852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:10.625972 16852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:10.636297 16852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=41309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:10.637719 16852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:10.639140 16852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:10.651793 16858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:10.651748 16862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:10.651748 16859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:10.652714 16852 server_base.cc:1034] running on GCE node
I20260811 06:24:10.653455 16852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:10.654901 16852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:10.656114 16852 hybrid_clock.cc:648] HybridClock initialized: now 1786429450656057 us; error 37 us; skew 500 ppm
I20260811 06:24:10.656436 16852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:10 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:10.659634 16852 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:24:10.661217 16852 webserver.cc:460] Webserver started at http://127.7.4.190:41309/ using document root <none> and password file <none>
I20260811 06:24:10.661947 16852 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:10.662106 16852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:10.667682 16852 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:24:10.670472 16870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:10.672008 16852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:10.672201 16852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "7135704df62b4332bd9ea2ca52c8a162"
format_stamp: "Formatted at 2026-08-11 06:24:07 on dist-test-slave-07c2"
server_key: "ae59a8e9b5d76231417a997f0211c7a0"
server_key_iv: "a5db009d4053e467c5a740cf55bfccbf"
server_key_version: "encryptionkey@0"
I20260811 06:24:10.673192 16852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:10.699214 16852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:10.703305 16852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:10.703604 16852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:10.723145 16852 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:34087
I20260811 06:24:10.723174 16925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:34087 every 8 connection(s)
I20260811 06:24:10.724828 16852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:10.727234 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16852
I20260811 06:24:10.728258 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:36539
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:34087
--webserver_port=46633
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:24:10.734589 16930 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:24:10.739904 16930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Bootstrap starting.
I20260811 06:24:10.759884 16930 log.cc:824] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:10.769028 16930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:24:10.769636 16930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Bootstrap complete.
I20260811 06:24:10.779471 16930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:10.780347 16930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7135704df62b4332bd9ea2ca52c8a162, State: Initialized, Role: FOLLOWER
I20260811 06:24:10.780997 16930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:10.781284 16930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:10.781401 16930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:10.781594 16930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:24:10.785538 16930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:10.786069 16930 leader_election.cc:302] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7135704df62b4332bd9ea2ca52c8a162; no voters:
I20260811 06:24:10.786763 16930 leader_election.cc:288] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:24:10.786896 16934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:24:10.788326 16934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [term 2 LEADER]: Becoming Leader. State: Replica: 7135704df62b4332bd9ea2ca52c8a162, State: Running, Role: LEADER
I20260811 06:24:10.788990 16934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } }
I20260811 06:24:10.789320 16930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:10.791525 16936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7135704df62b4332bd9ea2ca52c8a162. Latest consensus state: current_term: 2 leader_uuid: "7135704df62b4332bd9ea2ca52c8a162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } } }
I20260811 06:24:10.791905 16936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:10.792483 16935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7135704df62b4332bd9ea2ca52c8a162" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7135704df62b4332bd9ea2ca52c8a162" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34087 } } }
I20260811 06:24:10.792793 16935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:10.800805 16940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:10.808138 16940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:10.810027 16940 catalog_manager.cc:1259] Loaded cluster ID: a1294c6c8bdb4c5384b544f9cecd4eb5
I20260811 06:24:10.810237 16940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:10.815670 16940 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:10.817553 16940 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7135704df62b4332bd9ea2ca52c8a162: Loaded TSK: 0
I20260811 06:24:10.819585 16940 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:24:11.102936 16932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:11.103267 16932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:11.103451 16932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:11.110742 16932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:11.110894 16932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:11.111088 16932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:11.121037 16932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.129.keytab
--principal=oryx/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:36539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=46633
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:11.122381 16932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:11.123773 16932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:11.136956 16961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:11.136986 16963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:11.137559 16965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:11.138019 16932 server_base.cc:1034] running on GCE node
I20260811 06:24:11.138700 16932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:11.139748 16932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:11.140972 16932 hybrid_clock.cc:648] HybridClock initialized: now 1786429451140916 us; error 49 us; skew 500 ppm
I20260811 06:24:11.141317 16932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429451, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:11.144644 16932 init.cc:377] Logged in from keytab as oryx/127.7.4.129@KRBTEST.COM (short username oryx)
I20260811 06:24:11.146252 16932 webserver.cc:460] Webserver started at http://127.7.4.129:46633/ using document root <none> and password file <none>
I20260811 06:24:11.146937 16932 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:11.147089 16932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:11.152602 16932 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:11.155438 16972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.156991 16932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:11.157205 16932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "4601a32d6e20466b940c2d9f1f84b09e"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "b1f146c1bc96304f5724f2c4694a757a"
server_key_iv: "6ce6936f39832a0ef3a98e6b7873f8d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:11.158211 16932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:11.183938 16932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:11.188378 16932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:11.188685 16932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:11.189746 16932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:11.191184 16932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:11.191322 16932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.191457 16932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:11.191555 16932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.225057 16932 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:36539
I20260811 06:24:11.225144 17092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:36539 every 8 connection(s)
I20260811 06:24:11.226486 16932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:11.229475 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 16932
I20260811 06:24:11.230253 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:44359
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:34087
--webserver_port=37339
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:11.244825 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.229728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:36245)
0811 06:24:11.230637 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.230674 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.232517 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.233894 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.233906 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.234509 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.235031 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.235047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.236718 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.236723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.237905 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.237917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.238970 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.238978 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.239263 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.240005 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.240027 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.243933 (+ 3906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":390,"threads_started":1}
W20260811 06:24:11.245654 17093 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:34087 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.4.190:34087: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
W20260811 06:24:11.245685 17095 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.229448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:36245 (local address 127.7.4.190:34087)
0811 06:24:11.230257 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.230275 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.230827 (+ 552us) server_negotiation.cc:407] Connection header received
0811 06:24:11.233058 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.233070 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.233389 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.233740 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.235245 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.236536 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.238056 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.238719 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.238923 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.244479 (+ 5556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:36245: BlockingRecv error: recv got EOF from 127.7.4.129:36245 (error 108)
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":308,"threads_started":1}
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:11.255249 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.246667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:53837)
0811 06:24:11.246891 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.246912 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.247015 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.247646 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.247650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.247931 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.248066 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.248073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.249611 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.249615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.250491 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.250501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.251509 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.251513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.251638 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.252194 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.252212 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.255064 (+ 2852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 06:24:11.255671 17095 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.246882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:53837 (local address 127.7.4.190:34087)
0811 06:24:11.247316 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.247325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.247347 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:24:11.247400 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.247408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.247488 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.247585 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.248228 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.249464 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.250633 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.251378 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.251510 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.255407 (+ 3897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:53837: BlockingRecv error: recv got EOF from 127.7.4.129:53837 (error 108)
Metrics: {"server-negotiator.queue_time_us":248}
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:11.265177 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.255948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:56809)
0811 06:24:11.256170 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.256194 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.256315 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.256643 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.256650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.257090 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.257336 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.257351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.258339 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.258348 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.260089 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.260109 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.260796 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.260802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.261021 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.261707 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.261724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.264867 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 06:24:11.265326 17095 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.256243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:56809 (local address 127.7.4.190:34087)
0811 06:24:11.256381 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.256385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.256399 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:24:11.256436 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.256439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.256502 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.256585 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.257492 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.258185 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.260234 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.260638 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.260733 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.265056 (+ 4323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:56809: BlockingRecv error: recv got EOF from 127.7.4.129:56809 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 06:24:11.265615 17093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:24:11.677094 17097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:11.677524 17097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:11.677678 17097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:11.688575 17097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:11.688719 17097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:11.688922 17097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:11.703199 17097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.130.keytab
--principal=oryx/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:44359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=37339
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:11.704994 17097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:11.706830 17097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:11.725874 17103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:11.725841 17104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:11.727625 17106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:11.729800 17097 server_base.cc:1034] running on GCE node
I20260811 06:24:11.730846 17097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:11.732616 17097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:11.734136 17097 hybrid_clock.cc:648] HybridClock initialized: now 1786429451734038 us; error 103 us; skew 500 ppm
I20260811 06:24:11.734634 17097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429451, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:11.739811 17097 init.cc:377] Logged in from keytab as oryx/127.7.4.130@KRBTEST.COM (short username oryx)
I20260811 06:24:11.742074 17097 webserver.cc:460] Webserver started at http://127.7.4.130:37339/ using document root <none> and password file <none>
I20260811 06:24:11.743083 17097 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:11.743399 17097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:11.750790 17097 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 06:24:11.754676 17114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.756640 17097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 06:24:11.756887 17097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "8fc58aaebcc2484f909a5051c962cea6"
format_stamp: "Formatted at 2026-08-11 06:24:08 on dist-test-slave-07c2"
server_key: "d9573613f9e728f6a8508874a9e9b786"
server_key_iv: "b353a36368316183a39f537f7cce0436"
server_key_version: "encryptionkey@0"
I20260811 06:24:11.758234 17097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:11.782094 17097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:11.787810 17097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:11.788185 17097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:11.789516 17097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:11.791551 17097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:11.791720 17097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.791883 17097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:11.792007 17097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:11.835892 17097 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:44359
I20260811 06:24:11.836010 17238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:44359 every 8 connection(s)
I20260811 06:24:11.838277 17097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:11.845815 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17097
I20260811 06:24:11.846925 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:46403
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:34087
--webserver_port=34525
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:24:11.871814 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.842433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:52403 (local address 127.7.4.190:34087)
0811 06:24:11.843039 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.843047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.844658 (+ 1611us) server_negotiation.cc:407] Connection header received
0811 06:24:11.848844 (+ 4186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.848848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.848983 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.849157 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.853043 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.854199 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.859202 (+ 5003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.860030 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.860174 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.871119 (+ 10945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:52403: BlockingRecv error: recv got EOF from 127.7.4.130:52403 (error 108)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":311,"threads_started":1}
I20260811 06:24:11.872897 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.843425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:52403)
0811 06:24:11.844443 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.844501 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.848393 (+ 3892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.849556 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.849585 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.851031 (+ 1446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.852740 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.852785 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.854487 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.854495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.858873 (+ 4378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.858919 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.860439 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.860451 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.861115 (+ 664us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.862543 (+ 1428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.862632 (+ 89us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.870670 (+ 8038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":398,"threads_started":1}
W20260811 06:24:11.874012 17240 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:34087 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.4.190:34087: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:11.888085 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.875021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:45477)
0811 06:24:11.875355 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.875380 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.875545 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.876250 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.876259 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.876825 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.877106 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.877131 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.879231 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.879235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.880656 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.880673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.882446 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.882451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.882638 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.883595 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.883623 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.887826 (+ 4203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 06:24:11.888736 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.875157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:45477 (local address 127.7.4.190:34087)
0811 06:24:11.875575 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.875587 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.875635 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:24:11.875743 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.875752 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.875913 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.876112 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.877347 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.879054 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.880861 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.882215 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.882489 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.888247 (+ 5758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:45477: BlockingRecv error: recv got EOF from 127.7.4.130:45477 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:11 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:24:11.906291 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:11.889364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:37013 (local address 127.7.4.190:34087)
0811 06:24:11.889635 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:11.889641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:11.889695 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:24:11.889959 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:11.889962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:11.890041 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:11.890156 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:11.891785 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.892937 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.896643 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.897358 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.897498 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.906001 (+ 8503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:37013: BlockingRecv error: recv got EOF from 127.7.4.130:37013 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:24:11.906359 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:11.889300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:37013)
0811 06:24:11.889589 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:11.889620 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:11.889804 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:11.890253 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:11.890271 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:11.891089 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:11.891532 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.891562 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.893195 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.893214 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.896404 (+ 3190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:11.896436 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:11.897607 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:11.897629 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:11.898096 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:11.899141 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:11.899213 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:11.905763 (+ 6550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 06:24:11.907363 17240 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:12.281867 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.266827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:58085)
0811 06:24:12.267703 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:12.267742 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:12.268061 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:12.268833 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:12.268849 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:12.269734 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:12.270052 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.270070 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.271720 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.271740 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.273639 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.273660 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.275017 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.275027 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.275488 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.276546 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:12.276597 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:12.281547 (+ 4950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":516,"threads_started":1}
W20260811 06:24:12.282871 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:12.266926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:58085 (local address 127.7.4.190:34087)
0811 06:24:12.267263 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.267311 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.267917 (+ 606us) server_negotiation.cc:407] Connection header received
0811 06:24:12.268312 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.268326 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.268440 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.268773 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.270286 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.271502 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.273894 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.274632 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.274933 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.282263 (+ 7330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:58085: BlockingRecv error: recv got EOF from 127.7.4.129:58085 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 06:24:12.436532 17244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:12.436957 17244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:12.437089 17244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:12.450351 17244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:12.450623 17244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:12.450835 17244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:12.470333 17244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/krb5kdc/oryx_127.7.4.131.keytab
--principal=oryx/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:46403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=34525
--tserver_master_addrs=127.7.4.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:12.472939 17244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:12.475349 17244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:12.498304 17250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:12.498406 17251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:12.498391 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:12.500748 17244 server_base.cc:1034] running on GCE node
I20260811 06:24:12.501856 17244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:12.503527 17244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:12.504781 17244 hybrid_clock.cc:648] HybridClock initialized: now 1786429452504721 us; error 63 us; skew 500 ppm
I20260811 06:24:12.505373 17244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429452, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:12.510663 17244 init.cc:377] Logged in from keytab as oryx/127.7.4.131@KRBTEST.COM (short username oryx)
I20260811 06:24:12.513403 17244 webserver.cc:460] Webserver started at http://127.7.4.131:34525/ using document root <none> and password file <none>
I20260811 06:24:12.514613 17244 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:12.514904 17244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:12.524122 17244 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 06:24:12.528806 17263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:12.531217 17244 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:24:12.531564 17244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "c3563bcd2b0842dc80e7bdb76aa3134e"
format_stamp: "Formatted at 2026-08-11 06:24:09 on dist-test-slave-07c2"
server_key: "ad53c65f70b356e879cee144c2f93de9"
server_key_iv: "38493071efbc1927c95e508b186a8b7f"
server_key_version: "encryptionkey@0"
I20260811 06:24:12.533222 17244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:12.569207 17244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:12.577165 17244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:12.577662 17244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:12.579437 17244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:12.581812 17244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:12.582015 17244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:12.582237 17244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:12.582460 17244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:12.636372 17244 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:46403
I20260811 06:24:12.636502 17382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:46403 every 8 connection(s)
I20260811 06:24:12.638695 17244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:12.642743 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17244
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:24:12.674446 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:12.643399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:58545 (local address 127.7.4.190:34087)
0811 06:24:12.643759 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.643777 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.646419 (+ 2642us) server_negotiation.cc:407] Connection header received
0811 06:24:12.652761 (+ 6342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.652773 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.652868 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.652978 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.656419 (+ 3441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.657556 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.661521 (+ 3965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.662218 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.662511 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.673830 (+ 11319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:58545: BlockingRecv error: recv got EOF from 127.7.4.131:58545 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 06:24:12.675997 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.644179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:58545)
0811 06:24:12.646003 (+ 1824us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:12.646153 (+ 150us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:12.652046 (+ 5893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:12.653623 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:12.653649 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:12.654758 (+ 1109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:12.656038 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.656113 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.657798 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.657806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.661081 (+ 3275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.661113 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.662751 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.662766 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.663213 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.664595 (+ 1382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:12.664653 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:12.673477 (+ 8824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1186,"thread_start_us":726,"threads_started":1}
W20260811 06:24:12.677287 17383 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:34087 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.4.190:34087: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
I20260811 06:24:12.679551 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.644534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.7.4.190:34087)
0811 06:24:12.648206 (+ 3672us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.648225 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.648909 (+ 684us) server_negotiation.cc:407] Connection header received
0811 06:24:12.649345 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.649350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.649478 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.649619 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.654147 (+ 4528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.655486 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.658101 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.658802 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.658967 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.665169 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:12.665185 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:12.665229 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:12.665330 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:12.672917 (+ 7587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:12.674077 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:12.674093 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:12.674100 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:12.674583 (+ 483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:12.675485 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:12.675491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:12.675511 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:12.676346 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:12.676904 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:12.677853 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 06:24:12.678706 (+ 853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3406,"thread_start_us":3132,"threads_started":1}
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:12.692479 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.678421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:35797)
0811 06:24:12.678857 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:12.678881 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:12.679015 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:12.679563 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:12.679568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:12.680126 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:12.680402 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.680412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.682465 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.682475 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.684245 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.684284 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.685619 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.685628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.685963 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.686713 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:12.686744 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:12.691813 (+ 5069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292}
W20260811 06:24:12.692744 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:12.678520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:35797 (local address 127.7.4.190:34087)
0811 06:24:12.678917 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.678928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.678967 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:24:12.679153 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.679160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.679353 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.679546 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.680666 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.682222 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.684520 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.685372 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.685582 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.692233 (+ 6651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:35797: BlockingRecv error: recv got EOF from 127.7.4.131:35797 (error 108)
Metrics: {"server-negotiator.queue_time_us":205}
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:12.706264 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.693590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:44341)
0811 06:24:12.693852 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:12.693887 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:12.694050 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:12.694521 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:12.694524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:12.695063 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:12.695411 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.695439 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.697279 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.697291 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.699126 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.699140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.700460 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.700477 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.700740 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.701692 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:12.701729 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:12.705937 (+ 4208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 06:24:12.706718 17383 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:24:12.706959 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:12.693732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:44341 (local address 127.7.4.190:34087)
0811 06:24:12.693980 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.693992 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.694030 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:24:12.694213 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.694217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.694331 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.694451 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.695626 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.697054 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.699385 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.700186 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.700426 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.706244 (+ 5818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:44341: BlockingRecv error: recv got EOF from 127.7.4.131:44341 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:12 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:12.919435 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:12.907895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:38095)
0811 06:24:12.908389 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:12.908406 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:12.908579 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:12.908975 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:12.908981 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:12.909627 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:12.909941 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.909956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.911008 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.911012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.911989 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:12.912000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.912682 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.912686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:12.912820 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.916449 (+ 3629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:12.916470 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:12.919166 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":274,"threads_started":1}
W20260811 06:24:12.919727 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:12.907950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38095 (local address 127.7.4.190:34087)
0811 06:24:12.908212 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:12.908219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:12.908490 (+ 271us) server_negotiation.cc:407] Connection header received
0811 06:24:12.908726 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:12.908732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:12.908814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:12.908923 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:12.910100 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.910870 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.912124 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:12.912532 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:12.912634 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:12.919448 (+ 6814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:38095: BlockingRecv error: recv got EOF from 127.7.4.130:38095 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:13.291669 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:13.283015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:58839)
0811 06:24:13.283596 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:13.283614 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:13.283751 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:13.284066 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:13.284069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:13.284482 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:13.284627 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.284635 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.285612 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.285615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.286540 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.286552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.287217 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.287239 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.287427 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.288125 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:13.288148 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:13.291430 (+ 3282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":355,"threads_started":1}
W20260811 06:24:13.291849 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:13.283069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:58839 (local address 127.7.4.190:34087)
0811 06:24:13.283321 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:13.283331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:13.283700 (+ 369us) server_negotiation.cc:407] Connection header received
0811 06:24:13.283868 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:13.283872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:13.283946 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:13.284027 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:13.284750 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.285483 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.286670 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.287059 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.287187 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.291612 (+ 4425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:58839: BlockingRecv error: recv got EOF from 127.7.4.129:58839 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:13.719575 17390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:13.707422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:39591)
0811 06:24:13.710241 (+ 2819us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:13.710264 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:13.710434 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:13.710886 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:13.710891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:13.711583 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:13.711853 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.711868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.712943 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.712949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.713917 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.713928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.714755 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.714761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.714971 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.715712 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:13.715745 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:13.719348 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2650,"thread_start_us":2500,"threads_started":1}
W20260811 06:24:13.720043 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:13.707494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39591 (local address 127.7.4.190:34087)
0811 06:24:13.707821 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:13.707830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:13.710366 (+ 2536us) server_negotiation.cc:407] Connection header received
0811 06:24:13.710633 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:13.710639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:13.710726 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:13.710833 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:13.712017 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.712797 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.714068 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.714556 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.714701 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.719660 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:39591: BlockingRecv error: recv got EOF from 127.7.4.131:39591 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:13 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:13.929813 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:13.920447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:57817)
0811 06:24:13.921056 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:13.921072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:13.921239 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:13.921591 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:13.921595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:13.922015 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:13.922169 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.922178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.923435 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.923440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.924351 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:13.924363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.925092 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.925097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:13.925269 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.925964 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:13.925981 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:13.929445 (+ 3464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":375,"threads_started":1}
W20260811 06:24:13.929905 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:13.920490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:57817 (local address 127.7.4.190:34087)
0811 06:24:13.920684 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:13.920693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:13.921163 (+ 470us) server_negotiation.cc:407] Connection header received
0811 06:24:13.921367 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:13.921372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:13.921450 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:13.921553 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:13.922321 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.923248 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.924498 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:13.924930 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:13.925067 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:13.929652 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:57817: BlockingRecv error: recv got EOF from 127.7.4.130:57817 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:14.301455 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:14.292634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:44519)
0811 06:24:14.293179 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:14.293194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:14.293368 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:14.293720 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:14.293725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:14.294116 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:14.294290 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.294299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.295391 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.295401 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.296293 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.296304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.297030 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.297036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.297223 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.297834 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:14.297851 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:14.301056 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":278,"threads_started":1}
W20260811 06:24:14.301510 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:14.292818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44519 (local address 127.7.4.190:34087)
0811 06:24:14.293042 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:14.293048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:14.293289 (+ 241us) server_negotiation.cc:407] Connection header received
0811 06:24:14.293491 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:14.293496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:14.293582 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:14.293686 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:14.294448 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.295199 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.296440 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.296878 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.297014 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.301261 (+ 4247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:44519: BlockingRecv error: recv got EOF from 127.7.4.129:44519 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:14.728924 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:14.720605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:55023)
0811 06:24:14.721112 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:14.721129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:14.721258 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:14.721554 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:14.721557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:14.721978 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:14.722126 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.722134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.723192 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.723196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.724116 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.724126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.724873 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.724877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.724985 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.725644 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:14.725678 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:14.728686 (+ 3008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":287,"threads_started":1}
W20260811 06:24:14.729115 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:14.720687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:55023 (local address 127.7.4.190:34087)
0811 06:24:14.720929 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:14.720935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:14.721218 (+ 283us) server_negotiation.cc:407] Connection header received
0811 06:24:14.721362 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:14.721366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:14.721427 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:14.721506 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:14.722258 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.723064 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.724263 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.724741 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.724837 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.728870 (+ 4033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:55023: BlockingRecv error: recv got EOF from 127.7.4.131:55023 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:14 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:14.939566 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:14.931010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:60235)
0811 06:24:14.931564 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:14.931580 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:14.931704 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:14.931993 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:14.931996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:14.932325 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:14.932488 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.932495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.933458 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.933462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.934356 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:14.934368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.935006 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.935010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:14.935136 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.935820 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:14.935839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:14.939180 (+ 3341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":325,"threads_started":1}
W20260811 06:24:14.939670 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:14.931064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:60235 (local address 127.7.4.190:34087)
0811 06:24:14.931222 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:14.931227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:14.931653 (+ 426us) server_negotiation.cc:407] Connection header received
0811 06:24:14.931803 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:14.931807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:14.931867 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:14.931945 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:14.932612 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.933338 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.934481 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:14.934881 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:14.934968 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:14.939425 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:60235: BlockingRecv error: recv got EOF from 127.7.4.130:60235 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:15.311064 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:15.302549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:53997)
0811 06:24:15.303122 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:15.303139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:15.303269 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:15.303593 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:15.303597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:15.303975 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:15.304136 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.304143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.305163 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.305167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.306135 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.306146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.306797 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.306801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.306933 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.307671 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:15.307689 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:15.310697 (+ 3008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":320,"threads_started":1}
W20260811 06:24:15.311152 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:15.302675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:53997 (local address 127.7.4.190:34087)
0811 06:24:15.302857 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:15.302862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:15.303228 (+ 366us) server_negotiation.cc:407] Connection header received
0811 06:24:15.303403 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:15.303408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:15.303470 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:15.303544 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:15.304279 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.305033 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.306270 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.306671 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.306760 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.310904 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:53997: BlockingRecv error: recv got EOF from 127.7.4.129:53997 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:15.738498 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:15.729858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:58083)
0811 06:24:15.730391 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:15.730408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:15.730530 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:15.730834 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:15.730837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:15.731280 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:15.731500 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.731511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.732489 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.732493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.733389 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.733401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.734019 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.734023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.734143 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.734731 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:15.734747 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:15.738116 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":290,"threads_started":1}
W20260811 06:24:15.738559 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:15.729957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:58083 (local address 127.7.4.190:34087)
0811 06:24:15.730141 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:15.730145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:15.730484 (+ 339us) server_negotiation.cc:407] Connection header received
0811 06:24:15.730625 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:15.730629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:15.730688 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:15.730785 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:15.731647 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.732370 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.733511 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.733900 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.733988 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.738325 (+ 4337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:58083: BlockingRecv error: recv got EOF from 127.7.4.131:58083 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:15 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:15.949771 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:15.940668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:55889)
0811 06:24:15.941327 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:15.941350 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:15.941484 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:15.941891 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:15.941895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:15.942278 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:15.942432 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.942440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.943957 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.943961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.944914 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:15.944925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.945586 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.945590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:15.945729 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.946312 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:15.946329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:15.949550 (+ 3221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":281,"threads_started":1}
W20260811 06:24:15.949993 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:15.940734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:55889 (local address 127.7.4.190:34087)
0811 06:24:15.940931 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:15.940936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:15.941439 (+ 503us) server_negotiation.cc:407] Connection header received
0811 06:24:15.941628 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:15.941633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:15.941717 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:15.941831 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:15.942564 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.943829 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.945045 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:15.945450 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:15.945547 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:15.949749 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:55889: BlockingRecv error: recv got EOF from 127.7.4.130:55889 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:16.320783 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:16.312254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:52911)
0811 06:24:16.312761 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:16.312777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:16.312886 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:16.313155 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:16.313158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:16.313521 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:16.313717 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.313727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.315076 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.315081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.316126 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.316139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.316842 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.316846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.316961 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.317597 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:16.317615 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:16.320528 (+ 2913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":256,"threads_started":1}
W20260811 06:24:16.320967 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:16.312479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:52911 (local address 127.7.4.190:34087)
0811 06:24:16.312696 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:16.312702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:16.312851 (+ 149us) server_negotiation.cc:407] Connection header received
0811 06:24:16.312964 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:16.312968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:16.313027 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:16.313103 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:16.313853 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.314922 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.316262 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.316720 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.316808 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.320722 (+ 3914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:52911: BlockingRecv error: recv got EOF from 127.7.4.129:52911 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:16.747961 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:16.739644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:39127)
0811 06:24:16.740251 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:16.740266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:16.740466 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:16.740825 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:16.740830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:16.741279 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:16.741470 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.741480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.742548 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.742552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.743535 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.743547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.744221 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.744225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.744360 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.744919 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:16.744936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:16.747739 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":323,"threads_started":1}
W20260811 06:24:16.748198 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:16.739744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39127 (local address 127.7.4.190:34087)
0811 06:24:16.739972 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:16.739980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:16.740357 (+ 377us) server_negotiation.cc:407] Connection header received
0811 06:24:16.740604 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:16.740609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:16.740690 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:16.740791 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:16.741624 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.742395 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.743678 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.744091 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.744188 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.747938 (+ 3750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:39127: BlockingRecv error: recv got EOF from 127.7.4.131:39127 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:16 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:16.959116 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:16.950680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:60035)
0811 06:24:16.951234 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:16.951249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:16.951435 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:16.951785 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:16.951791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:16.952252 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:16.952412 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.952421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.953487 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.953493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.954405 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:16.954416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.955082 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.955087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:16.955258 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.955927 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:16.955945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:16.958765 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":269,"threads_started":1}
W20260811 06:24:16.959192 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:16.950811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:60035 (local address 127.7.4.190:34087)
0811 06:24:16.951041 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:16.951047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:16.951362 (+ 315us) server_negotiation.cc:407] Connection header received
0811 06:24:16.951563 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:16.951570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:16.951637 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:16.951741 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:16.952560 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.953313 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.954545 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:16.954940 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:16.955069 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:16.958951 (+ 3882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:60035: BlockingRecv error: recv got EOF from 127.7.4.130:60035 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:17.330171 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:17.321612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:42037)
0811 06:24:17.322222 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:17.322238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:17.322366 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:17.322669 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:17.322672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:17.323067 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:17.323247 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.323255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.324308 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.324312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.325212 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.325224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.325881 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.325885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.326004 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.326608 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:17.326626 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:17.329878 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":368,"threads_started":1}
W20260811 06:24:17.330348 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:17.321830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:42037 (local address 127.7.4.190:34087)
0811 06:24:17.322028 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:17.322032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:17.322316 (+ 284us) server_negotiation.cc:407] Connection header received
0811 06:24:17.322461 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:17.322465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:17.322525 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:17.322605 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:17.323437 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.324176 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.325337 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.325751 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.325837 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.330078 (+ 4241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:42037: BlockingRecv error: recv got EOF from 127.7.4.129:42037 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:17.756915 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:17.748899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:47357)
0811 06:24:17.749463 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:17.749479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:17.749592 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:17.749869 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:17.749872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:17.750274 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:17.750453 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.750462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.751467 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.751471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.752393 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.752404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.753059 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.753063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.753191 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.753740 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:17.753757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:17.756679 (+ 2922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":289,"threads_started":1}
W20260811 06:24:17.757076 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:17.748991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:47357 (local address 127.7.4.190:34087)
0811 06:24:17.749153 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:17.749157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:17.749552 (+ 395us) server_negotiation.cc:407] Connection header received
0811 06:24:17.749676 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:17.749679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:17.749745 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:17.749824 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:17.750580 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.751343 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.752514 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.752941 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.753030 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.756880 (+ 3850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:47357: BlockingRecv error: recv got EOF from 127.7.4.131:47357 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:17 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:17.975448 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:17.960312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:34477)
0811 06:24:17.960912 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:17.960929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:17.961103 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:17.961498 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:17.961503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:17.961994 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:17.962161 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.962172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.963502 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.963507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.964533 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:17.964549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.965474 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.965479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:17.965710 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.972027 (+ 6317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:17.972052 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:17.975107 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":340,"threads_started":1}
W20260811 06:24:17.975787 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:17.960430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:34477 (local address 127.7.4.190:34087)
0811 06:24:17.960708 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:17.960715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:17.961025 (+ 310us) server_negotiation.cc:407] Connection header received
0811 06:24:17.961261 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:17.961267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:17.961345 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:17.961455 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:17.962358 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.963275 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.964741 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:17.965259 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:17.965401 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:17.975492 (+ 10091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:34477: BlockingRecv error: recv got EOF from 127.7.4.130:34477 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:18.339663 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:18.331133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:38679)
0811 06:24:18.331879 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:18.331896 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:18.332016 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:18.332321 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:18.332325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:18.332738 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:18.332914 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.332923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.333959 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.333963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.334896 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.334908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.335593 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.335597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.335711 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.336253 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:18.336270 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:18.339276 (+ 3006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":309,"threads_started":1}
W20260811 06:24:18.339768 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:18.331183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:38679 (local address 127.7.4.190:34087)
0811 06:24:18.331410 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:18.331419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:18.331984 (+ 565us) server_negotiation.cc:407] Connection header received
0811 06:24:18.332118 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:18.332123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:18.332189 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:18.332265 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:18.333067 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.333804 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.335022 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.335460 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.335552 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.339476 (+ 3924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:38679: BlockingRecv error: recv got EOF from 127.7.4.129:38679 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:18.766196 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:18.757995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:38421)
0811 06:24:18.758485 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:18.758501 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:18.758616 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:18.758922 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:18.758925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:18.759372 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:18.759536 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.759545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.760525 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.760528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.761432 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.761443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.762221 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.762225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.762346 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.762989 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:18.763006 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:18.765859 (+ 2853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":277,"threads_started":1}
W20260811 06:24:18.766397 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:18.758038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:38421 (local address 127.7.4.190:34087)
0811 06:24:18.758286 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:18.758296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:18.758585 (+ 289us) server_negotiation.cc:407] Connection header received
0811 06:24:18.758734 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:18.758738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:18.758800 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:18.758879 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:18.759656 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.760408 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.761555 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.762086 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.762179 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.766128 (+ 3949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:38421: BlockingRecv error: recv got EOF from 127.7.4.131:38421 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:18 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:18.985369 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:18.976562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:44397)
0811 06:24:18.977386 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:18.977426 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:18.977597 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:18.977969 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:18.977974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:18.978404 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:18.978580 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.978589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.979697 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.979703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.980625 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:18.980638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.981398 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.981404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:18.981593 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.982160 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:18.982178 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:18.984991 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":334,"threads_started":1}
W20260811 06:24:18.985512 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:18.976618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:44397 (local address 127.7.4.190:34087)
0811 06:24:18.976859 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:18.976865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:18.977529 (+ 664us) server_negotiation.cc:407] Connection header received
0811 06:24:18.977745 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:18.977749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:18.977809 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:18.977926 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:18.978737 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.979547 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.980780 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:18.981225 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:18.981376 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:18.985249 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:44397: BlockingRecv error: recv got EOF from 127.7.4.130:44397 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:19.349162 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:19.340729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:50793)
0811 06:24:19.341262 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:19.341277 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:19.341430 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:19.341812 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:19.341816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:19.342253 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:19.342401 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.342409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.343552 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.343557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.344462 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.344473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.345256 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.345262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.345437 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.346073 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:19.346089 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:19.348830 (+ 2741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":305,"threads_started":1}
W20260811 06:24:19.349292 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:19.340802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:50793 (local address 127.7.4.190:34087)
0811 06:24:19.341009 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:19.341016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:19.341362 (+ 346us) server_negotiation.cc:407] Connection header received
0811 06:24:19.341571 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:19.341577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:19.341648 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:19.341758 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:19.342560 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.343407 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.344605 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.345064 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.345200 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.349040 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:50793: BlockingRecv error: recv got EOF from 127.7.4.129:50793 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:19.775733 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:19.767105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:46117)
0811 06:24:19.767886 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:19.767901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:19.768088 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:19.768524 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:19.768528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:19.769021 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:19.769196 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.769213 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.770256 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.770259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.771222 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.771232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.771867 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.771870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.771992 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.772563 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:19.772580 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:19.775487 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":393,"threads_started":1}
W20260811 06:24:19.775918 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:19.767393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:46117 (local address 127.7.4.190:34087)
0811 06:24:19.767611 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:19.767619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:19.768027 (+ 408us) server_negotiation.cc:407] Connection header received
0811 06:24:19.768228 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:19.768233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:19.768332 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:19.768420 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:19.769333 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.770124 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.771362 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.771737 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.771834 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.775669 (+ 3835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:46117: BlockingRecv error: recv got EOF from 127.7.4.131:46117 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:19 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:19.996336 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:19.986556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:42947)
0811 06:24:19.987171 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:19.987189 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:19.987385 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:19.987791 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:19.987797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:19.988287 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:19.988456 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.988466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.989960 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.989966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.991066 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:19.991081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.991956 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.991962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:19.992142 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.992801 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:19.992820 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:19.995939 (+ 3119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":345,"threads_started":1}
W20260811 06:24:19.996497 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:19.986658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:42947 (local address 127.7.4.190:34087)
0811 06:24:19.986863 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:19.986868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:19.987323 (+ 455us) server_negotiation.cc:407] Connection header received
0811 06:24:19.987542 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:19.987548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:19.987616 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:19.987731 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:19.988645 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.989794 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.991240 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:19.991741 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:19.991893 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:19.996210 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:42947: BlockingRecv error: recv got EOF from 127.7.4.130:42947 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 06:24:20 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:20 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:20.359567 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:20.350136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:52077)
0811 06:24:20.350671 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:20.350688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:20.350813 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:20.351106 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:20.351110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:20.351532 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:20.351738 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:20.351747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.352756 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.352759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:20.353786 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:20.353799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.354478 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.354483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:20.354599 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:20.355240 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:20.355265 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:20.359163 (+ 3898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":298,"threads_started":1}
W20260811 06:24:20.359591 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:20.350207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:52077 (local address 127.7.4.190:34087)
0811 06:24:20.350468 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:20.350474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:20.350777 (+ 303us) server_negotiation.cc:407] Connection header received
0811 06:24:20.350909 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:20.350913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:20.350975 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:20.351059 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:20.351870 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.352611 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.353908 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.354355 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.354444 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:20.359382 (+ 4938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:52077: BlockingRecv error: recv got EOF from 127.7.4.129:52077 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:24:20 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:20 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:20.785518 17411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:20.776811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:60255)
0811 06:24:20.777416 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:20.777433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:20.777553 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:20.777961 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:20.777966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:20.778388 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:20.778542 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:20.778550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.779598 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.779602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:20.780522 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:20.780533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.781378 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.781382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:20.781506 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:20.782101 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:20.782119 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:20.785164 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":355,"threads_started":1}
W20260811 06:24:20.785638 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:20.776829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:60255 (local address 127.7.4.190:34087)
0811 06:24:20.777080 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:20.777093 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:20.777505 (+ 412us) server_negotiation.cc:407] Connection header received
0811 06:24:20.777703 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:20.777710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:20.777787 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:20.777867 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:20.778677 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.779464 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.780757 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:20.781218 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:20.781313 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:20.785375 (+ 4062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:60255: BlockingRecv error: recv got EOF from 127.7.4.131:60255 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:21.005920 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:20.997581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:43267)
0811 06:24:20.998381 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:20.998398 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:20.998511 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:20.998813 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:20.998816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:20.999242 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:20.999453 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:20.999461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.000383 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.000386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.001322 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:21.001345 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.002139 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.002143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.002273 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.002863 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:21.002881 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:21.005674 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":551,"threads_started":1}
W20260811 06:24:21.006102 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:20.997617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:43267 (local address 127.7.4.190:34087)
0811 06:24:20.997831 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:20.997840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:20.998479 (+ 639us) server_negotiation.cc:407] Connection header received
0811 06:24:20.998608 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:20.998612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:20.998675 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:20.998763 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:20.999567 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.000259 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.001456 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.002007 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.002097 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.005873 (+ 3776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:43267: BlockingRecv error: recv got EOF from 127.7.4.130:43267 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:21.369305 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:21.360555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:33061)
0811 06:24:21.361097 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:21.361113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:21.361235 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:21.361553 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:21.361556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:21.362059 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:21.362273 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:21.362283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.363397 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.363405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.364756 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:21.364771 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.365540 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.365545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.365716 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.366279 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:21.366295 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:21.368972 (+ 2677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":318,"threads_started":1}
W20260811 06:24:21.369444 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:21.360602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:33061 (local address 127.7.4.190:34087)
0811 06:24:21.360800 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:21.360805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:21.361188 (+ 383us) server_negotiation.cc:407] Connection header received
0811 06:24:21.361330 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:21.361334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:21.361393 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:21.361505 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:21.362417 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.363220 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.364911 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.365387 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.365521 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.369166 (+ 3645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:33061: BlockingRecv error: recv got EOF from 127.7.4.129:33061 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:21 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:21.797672 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:21.786924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:54847)
0811 06:24:21.787596 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:21.787619 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:21.787775 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:21.788307 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:21.788314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:21.788923 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:21.789121 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:21.789130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.790748 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.790753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.792414 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:21.792443 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.793216 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.793222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:21.793414 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.794087 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:21.794105 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:21.797325 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":374,"threads_started":1}
W20260811 06:24:21.797744 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:21.786976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:54847 (local address 127.7.4.190:34087)
0811 06:24:21.787219 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:21.787226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:21.787719 (+ 493us) server_negotiation.cc:407] Connection header received
0811 06:24:21.787934 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:21.787979 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:21.788093 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:21.788227 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:21.789294 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.790603 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.792568 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:21.793064 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:21.793194 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:21.797526 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:54847: BlockingRecv error: recv got EOF from 127.7.4.131:54847 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:22.015539 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:22.006939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:60815)
0811 06:24:22.007528 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:22.007545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:22.007662 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:22.007952 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:22.007955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:22.008336 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:22.008482 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.008490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.009472 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.009475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.010411 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.010423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.011082 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.011086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.011208 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.011921 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:22.011940 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:22.015191 (+ 3251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":373,"threads_started":1}
W20260811 06:24:22.015693 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:22.007096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:60815 (local address 127.7.4.190:34087)
0811 06:24:22.007278 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:22.007325 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:22.007621 (+ 296us) server_negotiation.cc:407] Connection header received
0811 06:24:22.007764 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:22.007768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:22.007831 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:22.007905 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:22.008626 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.009361 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.010534 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.010952 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.011039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.015438 (+ 4399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:60815: BlockingRecv error: recv got EOF from 127.7.4.130:60815 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:22.379053 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:22.370449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:59875)
0811 06:24:22.371213 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:22.371240 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:22.371412 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:22.371722 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:22.371726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:22.372151 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:22.372327 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.372336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.373329 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.373333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.374190 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.374200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.374807 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.374811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.374921 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.375624 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:22.375642 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:22.378683 (+ 3041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":507,"threads_started":1}
W20260811 06:24:22.379142 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:22.370641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:59875 (local address 127.7.4.190:34087)
0811 06:24:22.370993 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:22.370999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:22.371340 (+ 341us) server_negotiation.cc:407] Connection header received
0811 06:24:22.371515 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:22.371519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:22.371586 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:22.371665 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:22.372468 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.373222 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.374308 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.374682 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.374768 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.378891 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:59875: BlockingRecv error: recv got EOF from 127.7.4.129:59875 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:22 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:22.806545 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:22.798571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:45169)
0811 06:24:22.799112 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:22.799128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:22.799242 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:22.799533 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:22.799536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:22.799931 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:22.800102 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.800110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.801079 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.801083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.802053 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:22.802064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.802824 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.802829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:22.802948 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.803642 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:22.803660 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:22.806253 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":314,"threads_started":1}
W20260811 06:24:22.806723 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:22.798760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:45169 (local address 127.7.4.190:34087)
0811 06:24:22.798952 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:22.798956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:22.799204 (+ 248us) server_negotiation.cc:407] Connection header received
0811 06:24:22.799349 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:22.799352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:22.799413 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:22.799493 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:22.800226 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.800972 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.802168 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:22.802685 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:22.802783 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:22.806404 (+ 3621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:45169: BlockingRecv error: recv got EOF from 127.7.4.131:45169 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:23.028640 17418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:23.016579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:58661)
0811 06:24:23.017108 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:23.017125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:23.017243 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:23.017569 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:23.017573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:23.018002 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:23.018171 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.018180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.019384 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.019392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.020339 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.020351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.021058 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.021064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.021268 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.025533 (+ 4265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:23.025550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:23.028354 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":299,"threads_started":1}
W20260811 06:24:23.028805 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:23.016781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:58661 (local address 127.7.4.190:34087)
0811 06:24:23.017008 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:23.017015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:23.017207 (+ 192us) server_negotiation.cc:407] Connection header received
0811 06:24:23.017341 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:23.017345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:23.017419 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:23.017496 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:23.018317 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.019227 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.020486 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.020899 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.021034 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.028556 (+ 7522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:58661: BlockingRecv error: recv got EOF from 127.7.4.130:58661 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:23.388885 17419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:23.380137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:40383)
0811 06:24:23.380722 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:23.380739 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:23.380897 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:23.381223 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:23.381227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:23.381656 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:23.381836 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.381844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.382884 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.382888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.384154 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.384170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.384960 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.384964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.385085 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.385723 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:23.385741 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:23.388655 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":292,"threads_started":1}
W20260811 06:24:23.389104 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:23.380218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:40383 (local address 127.7.4.190:34087)
0811 06:24:23.380445 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:23.380453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:23.380822 (+ 369us) server_negotiation.cc:407] Connection header received
0811 06:24:23.381020 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:23.381029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:23.381090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:23.381189 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:23.381975 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.382716 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.384307 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.384834 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.384929 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.388870 (+ 3941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:40383: BlockingRecv error: recv got EOF from 127.7.4.129:40383 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:23 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:23.816519 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:23.807879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:48555)
0811 06:24:23.808763 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:23.808781 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:23.808944 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:23.809347 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:23.809355 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:23.809857 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:23.810034 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.810043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.811001 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.811005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.811921 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:23.811932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.812574 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.812578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:23.812686 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.813338 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:23.813375 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:23.816176 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":672,"threads_started":1}
W20260811 06:24:23.816609 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:23.807979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:48555 (local address 127.7.4.190:34087)
0811 06:24:23.808320 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:23.808329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:23.808889 (+ 560us) server_negotiation.cc:407] Connection header received
0811 06:24:23.809088 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:23.809093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:23.809194 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:23.809276 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:23.810176 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.810879 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.812041 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:23.812453 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:23.812542 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:23.816371 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:48555: BlockingRecv error: recv got EOF from 127.7.4.131:48555 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:24.038247 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:24.029592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:41375)
0811 06:24:24.030181 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:24.030197 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:24.030360 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:24.030747 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:24.030752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:24.031168 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:24.031410 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.031421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.032530 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.032535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.033456 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.033467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.034234 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.034240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.034427 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.034998 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:24.035015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:24.037905 (+ 2890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":334,"threads_started":1}
W20260811 06:24:24.038507 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:24.029653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:41375 (local address 127.7.4.190:34087)
0811 06:24:24.029853 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:24.029858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:24.030292 (+ 434us) server_negotiation.cc:407] Connection header received
0811 06:24:24.030492 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:24.030497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:24.030571 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:24.030679 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:24.031560 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.032374 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.033615 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.034064 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.034218 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.038108 (+ 3890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:41375: BlockingRecv error: recv got EOF from 127.7.4.130:41375 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:24.398595 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:24.389975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:37975)
0811 06:24:24.390560 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:24.390576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:24.390739 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:24.391088 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:24.391093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:24.391539 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:24.391704 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.391723 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.392782 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.392787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.393733 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.393745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.394537 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.394544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.394718 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.395415 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:24.395433 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:24.398245 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":287,"threads_started":1}
W20260811 06:24:24.398712 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:24.390000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:37975 (local address 127.7.4.190:34087)
0811 06:24:24.390225 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:24.390233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:24.390661 (+ 428us) server_negotiation.cc:407] Connection header received
0811 06:24:24.390863 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:24.390869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:24.390944 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:24.391048 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:24.391863 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.392630 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.393890 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.394340 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.394473 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.398469 (+ 3996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:37975: BlockingRecv error: recv got EOF from 127.7.4.129:37975 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:24 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:24.826416 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:24.817560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:45385)
0811 06:24:24.818121 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:24.818137 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:24.818303 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:24.818641 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:24.818648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:24.819081 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:24.819245 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.819253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.820343 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.820349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.821269 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:24.821280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.822012 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.822017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:24.822200 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.822889 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:24.822911 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:24.826069 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":307,"threads_started":1}
W20260811 06:24:24.826547 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:24.817639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:45385 (local address 127.7.4.190:34087)
0811 06:24:24.817910 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:24.817918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:24.818244 (+ 326us) server_negotiation.cc:407] Connection header received
0811 06:24:24.818433 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:24.818437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:24.818497 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:24.818599 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:24.819440 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.820183 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.821440 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:24.821870 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:24.821998 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:24.826274 (+ 4276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:45385: BlockingRecv error: recv got EOF from 127.7.4.131:45385 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:25.047385 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:25.039215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:40329)
0811 06:24:25.039897 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:25.039914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:25.040029 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:25.040331 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:25.040335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:25.040773 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:25.040934 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.040943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.041902 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.041906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.042809 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.042820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.043597 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.043603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.043728 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.044292 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:25.044309 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:25.047070 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":409,"threads_started":1}
W20260811 06:24:25.047539 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:25.039413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:40329 (local address 127.7.4.190:34087)
0811 06:24:25.039592 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:25.039596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:25.040018 (+ 422us) server_negotiation.cc:407] Connection header received
0811 06:24:25.040137 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:25.040140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:25.040204 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:25.040323 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:25.041063 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.041776 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.042932 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.043366 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.043505 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.047256 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:40329: BlockingRecv error: recv got EOF from 127.7.4.130:40329 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:25.408327 17426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:25.399682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:34219)
0811 06:24:25.400203 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:25.400228 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:25.400362 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:25.400646 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:25.400648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:25.401082 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:25.401367 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.401380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.402641 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.402646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.403636 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.403649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.404415 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.404421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.404594 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.405168 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:25.405186 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:25.408065 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":260,"threads_started":1}
W20260811 06:24:25.408489 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:25.399743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:34219 (local address 127.7.4.190:34087)
0811 06:24:25.399898 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:25.399902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:25.400316 (+ 414us) server_negotiation.cc:407] Connection header received
0811 06:24:25.400455 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:25.400459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:25.400520 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:25.400600 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:25.401516 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.402467 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.403789 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.404260 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.404393 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.408252 (+ 3859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:34219: BlockingRecv error: recv got EOF from 127.7.4.129:34219 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:25 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:25.835912 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:25.827494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:45057)
0811 06:24:25.828070 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:25.828085 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:25.828241 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:25.828632 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:25.828636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:25.829064 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:25.829227 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.829236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.830257 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.830262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.831201 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:25.831212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.832048 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.832054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:25.832228 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.832826 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:25.832843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:25.835577 (+ 2734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":245,"threads_started":1}
W20260811 06:24:25.835978 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:25.827611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:45057 (local address 127.7.4.190:34087)
0811 06:24:25.827994 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:25.828000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:25.828168 (+ 168us) server_negotiation.cc:407] Connection header received
0811 06:24:25.828390 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:25.828393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:25.828452 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:25.828562 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:25.829374 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.830118 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.831402 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:25.831867 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:25.832009 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:25.835769 (+ 3760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:45057: BlockingRecv error: recv got EOF from 127.7.4.131:45057 (error 108)
Metrics: {"server-negotiator.queue_time_us":233}
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:26.057183 17428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:26.048433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:40833)
0811 06:24:26.049007 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:26.049057 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:26.049218 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:26.049642 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:26.049649 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:26.050230 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:26.050514 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.050528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.051722 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.051726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.052722 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.052735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.053383 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.053387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.053506 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.054107 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:26.054124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:26.056915 (+ 2791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":319,"threads_started":1}
W20260811 06:24:26.057334 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:26.048603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:40833 (local address 127.7.4.190:34087)
0811 06:24:26.048780 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:26.048784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:26.049139 (+ 355us) server_negotiation.cc:407] Connection header received
0811 06:24:26.049382 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:26.049390 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:26.049478 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:26.049564 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:26.050701 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.051584 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.052850 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.053252 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.053337 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.057095 (+ 3758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:40833: BlockingRecv error: recv got EOF from 127.7.4.130:40833 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:26.418176 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:26.409519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:33159)
0811 06:24:26.410032 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:26.410048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:26.410171 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:26.410502 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:26.410505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:26.410908 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:26.411101 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.411109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.412124 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.412128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.413388 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.413404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.414297 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.414301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.414423 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.415000 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:26.415017 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:26.417925 (+ 2908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":289,"threads_started":1}
W20260811 06:24:26.418354 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:26.409601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:33159 (local address 127.7.4.190:34087)
0811 06:24:26.409854 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:26.409861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:26.410126 (+ 265us) server_negotiation.cc:407] Connection header received
0811 06:24:26.410284 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:26.410287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:26.410372 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:26.410452 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:26.411229 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.412003 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.413539 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.414156 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.414259 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.418105 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:33159: BlockingRecv error: recv got EOF from 127.7.4.129:33159 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:26 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.131@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:26.845467 17430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:26.836935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.131:33137)
0811 06:24:26.837462 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:26.837479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:26.837656 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:26.838083 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:26.838088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:26.838496 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:26.838686 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.838695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.840178 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.840182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.841248 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:26.841260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.841900 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.841903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:26.842021 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.842603 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:26.842620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:26.845226 (+ 2606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":285,"threads_started":1}
W20260811 06:24:26.845638 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:26.836986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:33137 (local address 127.7.4.190:34087)
0811 06:24:26.837191 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:26.837197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:26.837571 (+ 374us) server_negotiation.cc:407] Connection header received
0811 06:24:26.837804 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:26.837810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:26.837889 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:26.837992 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:26.838830 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.840049 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.841365 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:26.841782 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:26.841874 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:26.845410 (+ 3536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.131:33137: BlockingRecv error: recv got EOF from 127.7.4.131:33137 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 06:24:27 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:27 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.130@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:27.067274 17431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:27.058564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.130:38211)
0811 06:24:27.059059 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:27.059075 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:27.059191 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:27.059548 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:27.059551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:27.059988 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:27.060196 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:27.060206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.061171 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.061174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:27.062165 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:27.062176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.063051 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.063058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:27.063264 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.063983 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:27.064008 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:27.067017 (+ 3009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":273,"threads_started":1}
W20260811 06:24:27.067482 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:27.058615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38211 (local address 127.7.4.190:34087)
0811 06:24:27.058863 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:27.058870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:27.059159 (+ 289us) server_negotiation.cc:407] Connection header received
0811 06:24:27.059307 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:27.059311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:27.059403 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:27.059501 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:27.060329 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.061055 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.062300 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.062872 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.063040 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.067221 (+ 4181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.130:38211: BlockingRecv error: recv got EOF from 127.7.4.130:38211 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:24:27 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:24:27 dist-test-slave-07c2 krb5kdc[16279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.4.129@KRBTEST.COM for oryx/127.7.4.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:24:27.427045 17432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:27.419148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34087 (local address 127.7.4.129:56635)
0811 06:24:27.419734 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:27.419751 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:27.419863 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:27.420166 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:27.420170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:27.420581 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:27.420763 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:27.420771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.421772 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.421776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:27.422694 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:27.422706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.423434 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.423438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:27.423570 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.424114 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:27.424130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:27.426831 (+ 2701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.4.190:34087: Server oryx/127.7.4.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":319,"threads_started":1}
W20260811 06:24:27.427266 17242 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:27.419223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:56635 (local address 127.7.4.190:34087)
0811 06:24:27.419454 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:27.419459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:27.419837 (+ 378us) server_negotiation.cc:407] Connection header received
0811 06:24:27.419966 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:27.419969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:27.420031 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:27.420108 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:27.420888 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.421632 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.422823 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.423239 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.423359 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.427038 (+ 3679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.4.129:56635: BlockingRecv error: recv got EOF from 127.7.4.129:56635 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 06:24:27.654048 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16932
I20260811 06:24:27.666627 17435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:27.655166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.7.4.129:36539)
0811 06:24:27.655976 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:27.655985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:27.656016 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:24:27.656074 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:27.656081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:27.656309 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:27.656461 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:27.657854 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.658931 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.660130 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.660732 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.660870 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.663537 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:27.663545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:27.663558 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:27.663595 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:27.666112 (+ 2517us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:24:27.666388 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35974: Request ticket server kudu/127.7.4.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":465,"threads_started":1}
W20260811 06:24:27.666752 17435 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35974: Request ticket server kudu/127.7.4.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:24:27.666888 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16932: Not authorized: Client connection negotiation failed: client connection to 127.7.4.129:36539: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:24:27.667100 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16932
I20260811 06:24:27.691381 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17097
I20260811 06:24:27.703400 17436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:27.692409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.7.4.130:44359)
0811 06:24:27.692966 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:27.692973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:27.693000 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:24:27.693062 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:27.693068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:27.693295 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:27.693447 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:27.694774 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.695848 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.697010 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.697561 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.697660 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.700120 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:27.700128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:27.700143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:27.700182 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:27.702868 (+ 2686us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:24:27.703106 (+ 238us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36142: Request ticket server kudu/127.7.4.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":256,"threads_started":1}
W20260811 06:24:27.703513 17436 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36142: Request ticket server kudu/127.7.4.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:24:27.703562 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17097: Not authorized: Client connection negotiation failed: client connection to 127.7.4.130:44359: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:24:27.703754 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17097
I20260811 06:24:27.726537 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17244
I20260811 06:24:27.738521 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:27.727519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.7.4.131:46403)
0811 06:24:27.728166 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:27.728173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:27.728201 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:24:27.728267 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:27.728273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:27.728495 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:27.728675 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:27.729999 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.731121 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.732391 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:27.732940 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:27.733035 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:27.735436 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:27.735444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:27.735458 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:27.735491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:27.738029 (+ 2538us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:24:27.738289 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50696: Request ticket server kudu/127.7.4.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":344,"threads_started":1}
W20260811 06:24:27.738673 17437 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50696: Request ticket server kudu/127.7.4.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:24:27.738654 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17244: Not authorized: Client connection negotiation failed: client connection to 127.7.4.131:46403: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.4.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:24:27.738817 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17244
I20260811 06:24:27.763362 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 16852
I20260811 06:24:27.764771 16919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:27.883868 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 16852
2026-08-11T06:24:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21865 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T06:24:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:27Z Disabled control of system clock
I20260811 06:24:27.976820 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37441
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:24:28.307876 17442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:28.308136 17442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:28.308274 17442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:28.315397 17442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:28.315486 17442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:28.315573 17442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:28.315670 17442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:28.315752 17442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:28.324666 17442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:28.325927 17442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:28.327641 17442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:28.339351 17448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:28.339282 17449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:28.339385 17451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:28.340152 17442 server_base.cc:1034] running on GCE node
I20260811 06:24:28.340926 17442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:28.342235 17442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:28.343454 17442 hybrid_clock.cc:648] HybridClock initialized: now 1786429468343417 us; error 47 us; skew 500 ppm
I20260811 06:24:28.343796 17442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:28.349941 17442 webserver.cc:460] Webserver started at http://127.7.4.190:34721/ using document root <none> and password file <none>
I20260811 06:24:28.350647 17442 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:28.350785 17442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:28.351123 17442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:28.354041 17442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "e7a62ba52fe8445784ffb826bcfe8a39"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "9d25f58e72a4d9fd62be68f33905b7f4"
server_key_iv: "edf22ec5008155c4ffb474851a5d7e5c"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.354835 17442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "e7a62ba52fe8445784ffb826bcfe8a39"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "9d25f58e72a4d9fd62be68f33905b7f4"
server_key_iv: "edf22ec5008155c4ffb474851a5d7e5c"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.359826 17442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 06:24:28.363041 17458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:28.364315 17442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:24:28.364494 17442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "e7a62ba52fe8445784ffb826bcfe8a39"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "9d25f58e72a4d9fd62be68f33905b7f4"
server_key_iv: "edf22ec5008155c4ffb474851a5d7e5c"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.364712 17442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:28.384985 17442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:28.385772 17442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:28.386061 17442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:28.404551 17442 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:37441
I20260811 06:24:28.404572 17514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:37441 every 8 connection(s)
I20260811 06:24:28.405845 17442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:28.409516 17515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:28.409662 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17442
I20260811 06:24:28.409902 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:24:28.410789 7186 external_mini_cluster.cc:1428] Setting key b70fdfa4588ef3d7489442d9132f9dde
I20260811 06:24:28.423216 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39: Bootstrap starting.
I20260811 06:24:28.424757 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:28.412971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.7.4.190:37441)
0811 06:24:28.413986 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:28.413998 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:28.414048 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:24:28.415993 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:28.416026 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:28.416468 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:28.416782 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:28.417401 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.418606 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.420531 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.421215 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.421350 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:28.422596 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:28.422602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:28.422614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:28.423115 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:28.423200 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:28.423220 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:28.423357 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:28.423563 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 06:24:28.423868 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":382,"threads_started":1}
I20260811 06:24:28.426357 17515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:28.427477 17515 log.cc:824] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:28.429956 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39: No bootstrap required, opened a new log
I20260811 06:24:28.436625 17515 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } }
I20260811 06:24:28.436959 17515 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:28.437079 17515 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a62ba52fe8445784ffb826bcfe8a39, State: Initialized, Role: FOLLOWER
I20260811 06:24:28.437659 17515 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } }
I20260811 06:24:28.437881 17515 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:28.438032 17515 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:28.438234 17515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:28.440487 17515 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } }
I20260811 06:24:28.440961 17515 leader_election.cc:302] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7a62ba52fe8445784ffb826bcfe8a39; no voters:
I20260811 06:24:28.441555 17515 leader_election.cc:288] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:24:28.441730 17520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:28.442492 17520 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [term 1 LEADER]: Becoming Leader. State: Replica: e7a62ba52fe8445784ffb826bcfe8a39, State: Running, Role: LEADER
I20260811 06:24:28.443089 17520 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } }
I20260811 06:24:28.443843 17515 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:28.445957 17521 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } } }
I20260811 06:24:28.446374 17521 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:28.446199 17522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7a62ba52fe8445784ffb826bcfe8a39. Latest consensus state: current_term: 1 leader_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7a62ba52fe8445784ffb826bcfe8a39" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37441 } } }
I20260811 06:24:28.446506 17522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:28.448558 17529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:28.454147 17529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:28.462934 17529 catalog_manager.cc:1347] Generated new cluster ID: fb2bcd6cce674c29ae7c2441abe923f7
I20260811 06:24:28.463042 17529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:28.490432 17529 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:24:28.492338 17529 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:28.501773 17529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7a62ba52fe8445784ffb826bcfe8a39: Generated new TSK 0
I20260811 06:24:28.502607 17529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:24:28.507253 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--builtin_ntp_servers=127.7.4.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 06:24:28.871973 17541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:28.872226 17541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:28.872308 17541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:28.879026 17541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:28.879112 17541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:28.879208 17541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:28.887727 17541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:28.888801 17541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:28.890693 17541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:28.903169 17547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:28.903146 17546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:28.903615 17549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:28.904029 17541 server_base.cc:1034] running on GCE node
I20260811 06:24:28.904621 17541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:28.905509 17541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:28.906733 17541 hybrid_clock.cc:648] HybridClock initialized: now 1786429468906694 us; error 54 us; skew 500 ppm
I20260811 06:24:28.907033 17541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:28.909329 17541 webserver.cc:460] Webserver started at http://127.7.4.129:38217/ using document root <none> and password file <none>
I20260811 06:24:28.910007 17541 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:28.910172 17541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:28.910499 17541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:28.913226 17541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "666cba6a12a742108ce2ce38bf228056"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "4216902f148bc317602f4cf777d87866"
server_key_iv: "e666aa5893d192065020f0bf47137b5a"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.914009 17541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "666cba6a12a742108ce2ce38bf228056"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "4216902f148bc317602f4cf777d87866"
server_key_iv: "e666aa5893d192065020f0bf47137b5a"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.918817 17541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 06:24:28.922061 17555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:28.923267 17541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:28.923504 17541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "666cba6a12a742108ce2ce38bf228056"
format_stamp: "Formatted at 2026-08-11 06:24:28 on dist-test-slave-07c2"
server_key: "4216902f148bc317602f4cf777d87866"
server_key_iv: "e666aa5893d192065020f0bf47137b5a"
server_key_version: "encryptionkey@0"
I20260811 06:24:28.923719 17541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:28.937829 17541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:28.938603 17541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:28.938869 17541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:28.939822 17541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:28.941198 17541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:28.941332 17541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:28.941453 17541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:28.941540 17541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:28.974987 17541 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:38423
I20260811 06:24:28.975037 17678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:38423 every 8 connection(s)
I20260811 06:24:28.976341 17541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:28.981523 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17541
I20260811 06:24:28.981736 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:24:28.982395 7186 external_mini_cluster.cc:1428] Setting key 683cba053ea1e93d4a0566dd5df2524c
I20260811 06:24:28.985144 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--builtin_ntp_servers=127.7.4.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:24:28.989372 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:28.978903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44425 (local address 127.7.4.190:37441)
0811 06:24:28.979394 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:28.979399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:28.980564 (+ 1165us) server_negotiation.cc:407] Connection header received
0811 06:24:28.982480 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:28.982486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:28.982577 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:28.982715 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:28.983972 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.984880 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.986492 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.986905 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.987041 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:28.988484 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:28.988491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:28.988495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:28.988521 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:28.988571 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:28.988584 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:28.988689 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:28.988926 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:24:28.989022 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":220,"threads_started":1}
I20260811 06:24:28.990293 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:28.979520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37441 (local address 127.7.4.129:44425)
0811 06:24:28.980390 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:28.980425 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:28.982184 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:28.982943 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:28.982952 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:28.983030 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:28.983792 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:28.983809 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.985050 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.985054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:28.986339 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:28.986352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:28.987137 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:28.987145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:28.987457 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:28.988178 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:28.988181 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:28.988218 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:28.988288 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:28.988691 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:28.988706 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:28.988717 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:24:28.988890 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 06:24:28.989262 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":282,"threads_started":1}
I20260811 06:24:28.991672 17679 heartbeater.cc:347] Connected to a master server at 127.7.4.190:37441
I20260811 06:24:28.991976 17679 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:28.992866 17679 heartbeater.cc:511] Master 127.7.4.190:37441 requested a full tablet report, sending...
I20260811 06:24:28.994923 17476 ts_manager.cc:194] Registered new tserver with Master: 666cba6a12a742108ce2ce38bf228056 (127.7.4.129:38423)
I20260811 06:24:28.996449 17476 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.129:44425
W20260811 06:24:29.315989 17683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:29.316253 17683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:29.316337 17683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:29.323376 17683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:29.323474 17683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:29.323577 17683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:29.332500 17683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:29.333678 17683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:29.335603 17683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:29.348507 17690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:29.348600 17691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:29.348637 17693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:29.350028 17683 server_base.cc:1034] running on GCE node
I20260811 06:24:29.350553 17683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:29.351491 17683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:29.352710 17683 hybrid_clock.cc:648] HybridClock initialized: now 1786429469352650 us; error 53 us; skew 500 ppm
I20260811 06:24:29.353052 17683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:29.355258 17683 webserver.cc:460] Webserver started at http://127.7.4.130:41961/ using document root <none> and password file <none>
I20260811 06:24:29.356061 17683 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:29.356178 17683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:29.356508 17683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:29.359258 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "0f58e9bde0b64162b8442394cfdf053a"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "ab70b86b21dca58e08307ffa21464842"
server_key_iv: "449b72a1e4e73bb187535d80c55dbd44"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.360128 17683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "0f58e9bde0b64162b8442394cfdf053a"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "ab70b86b21dca58e08307ffa21464842"
server_key_iv: "449b72a1e4e73bb187535d80c55dbd44"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.365175 17683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:24:29.368520 17699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.369946 17683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:29.370122 17683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "0f58e9bde0b64162b8442394cfdf053a"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "ab70b86b21dca58e08307ffa21464842"
server_key_iv: "449b72a1e4e73bb187535d80c55dbd44"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.370333 17683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:29.384151 17683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:29.384924 17683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:29.385195 17683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:29.386094 17683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:29.387705 17683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:29.387832 17683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.387964 17683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:29.388065 17683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.423213 17683 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:35717
I20260811 06:24:29.423336 17825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:35717 every 8 connection(s)
I20260811 06:24:29.424520 17683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:29.434710 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17683
I20260811 06:24:29.434897 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:24:29.435400 7186 external_mini_cluster.cc:1428] Setting key 815a92410bf68fa4221a55d00b6c6268
I20260811 06:24:29.436959 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:29.426876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:44003 (local address 127.7.4.190:37441)
0811 06:24:29.427076 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:29.427081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:29.428687 (+ 1606us) server_negotiation.cc:407] Connection header received
0811 06:24:29.430414 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:29.430419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:29.430514 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:29.430631 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:29.431711 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.432431 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.434017 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.434396 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.434527 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:29.436068 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:29.436075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:29.436079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:29.436100 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:29.436138 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:29.436153 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:29.436254 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:29.436545 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 06:24:29.436661 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:24:29.437937 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:29.427442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37441 (local address 127.7.4.130:44003)
0811 06:24:29.428474 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:29.428524 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:29.430135 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:29.430859 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:29.430867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:29.430946 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:29.431541 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:29.431559 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.432608 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.432612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:29.433878 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:29.433890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.434609 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.434616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:29.434886 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:29.435704 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:29.435707 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:29.435753 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:29.435849 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:29.436286 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:29.436299 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:29.436308 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:24:29.436488 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 06:24:29.436803 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":345,"threads_started":1}
I20260811 06:24:29.438644 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--builtin_ntp_servers=127.7.4.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:24:29.439188 17826 heartbeater.cc:347] Connected to a master server at 127.7.4.190:37441
I20260811 06:24:29.439497 17826 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:29.440191 17826 heartbeater.cc:511] Master 127.7.4.190:37441 requested a full tablet report, sending...
I20260811 06:24:29.441495 17476 ts_manager.cc:194] Registered new tserver with Master: 0f58e9bde0b64162b8442394cfdf053a (127.7.4.130:35717)
I20260811 06:24:29.442312 17476 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.130:44003
W20260811 06:24:29.779313 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:29.779564 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:29.779712 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:29.786720 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:29.786844 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:29.787027 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:29.795894 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:37441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:29.797116 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:29.798894 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:29.811918 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:29.811970 17835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:29.812000 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:29.813249 17830 server_base.cc:1034] running on GCE node
I20260811 06:24:29.813848 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:29.814718 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:29.815909 17830 hybrid_clock.cc:648] HybridClock initialized: now 1786429469815874 us; error 34 us; skew 500 ppm
I20260811 06:24:29.816242 17830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:29.818493 17830 webserver.cc:460] Webserver started at http://127.7.4.131:41075/ using document root <none> and password file <none>
I20260811 06:24:29.819165 17830 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:29.819284 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:29.819664 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:29.822361 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "a19484aa97854828b3bb4fed37059e8f"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "7635eada5753ed5b5e47ec765704cde2"
server_key_iv: "b79fb75eda7aa633d2534907faa5e8c0"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.823173 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "a19484aa97854828b3bb4fed37059e8f"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "7635eada5753ed5b5e47ec765704cde2"
server_key_iv: "b79fb75eda7aa633d2534907faa5e8c0"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.828327 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:24:29.831543 17844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.832798 17830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:24:29.832975 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "a19484aa97854828b3bb4fed37059e8f"
format_stamp: "Formatted at 2026-08-11 06:24:29 on dist-test-slave-07c2"
server_key: "7635eada5753ed5b5e47ec765704cde2"
server_key_iv: "b79fb75eda7aa633d2534907faa5e8c0"
server_key_version: "encryptionkey@0"
I20260811 06:24:29.833209 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:29.861516 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:29.862404 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:29.862685 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:29.863612 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:29.865154 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:29.865278 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.865417 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:29.865512 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:29.898622 17830 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:34669
I20260811 06:24:29.898727 17964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:34669 every 8 connection(s)
I20260811 06:24:29.899899 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:29.909029 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17830
I20260811 06:24:29.909221 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:24:29.909796 7186 external_mini_cluster.cc:1428] Setting key 5c1fc0f07d79c771746dc65c7d2ee7c8
I20260811 06:24:29.912067 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:29.902411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:47845 (local address 127.7.4.190:37441)
0811 06:24:29.902605 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:29.902609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:29.903897 (+ 1288us) server_negotiation.cc:407] Connection header received
0811 06:24:29.905570 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:29.905574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:29.905638 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:29.905727 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:29.906633 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.907402 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.909024 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.909538 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.909633 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:29.911203 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:29.911208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:29.911211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:29.911228 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:29.911257 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:29.911269 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:29.911391 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:29.911607 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 06:24:29.911723 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:24:29.912974 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:29.902827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37441 (local address 127.7.4.131:47845)
0811 06:24:29.903773 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:29.903806 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:29.905352 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:29.905905 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:29.905912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:29.905992 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:29.906498 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:29.906512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.907526 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.907530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:29.908863 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:29.908881 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.909769 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.909778 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:29.910064 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:29.910894 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:29.910897 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:29.910936 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:29.911001 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:29.911393 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:29.911408 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:29.911417 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:24:29.911601 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 06:24:29.911947 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":305,"threads_started":1}
I20260811 06:24:29.914130 17965 heartbeater.cc:347] Connected to a master server at 127.7.4.190:37441
I20260811 06:24:29.914397 17965 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:29.915094 17965 heartbeater.cc:511] Master 127.7.4.190:37441 requested a full tablet report, sending...
I20260811 06:24:29.916421 17476 ts_manager.cc:194] Registered new tserver with Master: a19484aa97854828b3bb4fed37059e8f (127.7.4.131:34669)
I20260811 06:24:29.917193 17476 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.131:47845
I20260811 06:24:29.924912 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:24:29.931062 17681 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:29.929512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.7.4.190:37441)
0811 06:24:29.929717 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:29.929724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:29.930166 (+ 442us) server_negotiation.cc:407] Connection header received
0811 06:24:29.930285 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:29.930288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:29.930352 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:29.930426 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:29.930863 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49892: BlockingRecv error: recv got EOF from 127.0.0.1:49892 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:24:29.931941 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17541
I20260811 06:24:29.939697 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:29.932826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.7.4.129:38423)
0811 06:24:29.933408 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:29.933414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:29.933439 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:24:29.933505 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:29.933511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:29.933679 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:29.933800 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:29.934112 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.935138 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.936594 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:29.937155 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:29.937304 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:29.938332 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:29.938339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:29.938349 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:29.938882 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:29.938928 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:29.938947 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:29.939070 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:29.939228 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 06:24:29.939388 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":304,"threads_started":1}
I20260811 06:24:29.940753 17674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:30.020983 17679 heartbeater.cc:503] Master 127.7.4.190:37441 was elected leader, sending a full tablet report...
I20260811 06:24:30.022125 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17541
W20260811 06:24:30.042392 17462 connection.cc:535] server connection from 127.7.4.129:44425 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:24:30.042759 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17683
I20260811 06:24:30.050472 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:30.043713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.7.4.130:35717)
0811 06:24:30.044293 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:30.044307 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:30.044334 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:24:30.044397 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:30.044404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:30.044605 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:30.044746 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:30.045088 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:30.046053 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:30.047175 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:30.047958 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:30.048068 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:30.049063 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:30.049071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:30.049083 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:30.049677 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:30.049737 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:30.049756 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:30.049853 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:30.049988 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 06:24:30.050138 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":312,"threads_started":1}
I20260811 06:24:30.051654 17821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:30.133697 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17683
I20260811 06:24:30.154294 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17830
I20260811 06:24:30.162338 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:30.155248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.7.4.131:34669)
0811 06:24:30.155929 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:30.155936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:30.155965 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:24:30.156024 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:30.156031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:30.156255 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:30.156372 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:30.156786 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:30.157820 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:30.159028 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:30.159810 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:30.159903 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:30.160899 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:30.160907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:30.160920 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:30.161485 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:30.161530 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:30.161549 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:30.161649 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:30.161765 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 06:24:30.161916 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":349,"threads_started":1}
I20260811 06:24:30.163569 17960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:30.243078 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17830
I20260811 06:24:30.263803 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17442
I20260811 06:24:30.264979 17509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:30.367774 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17442
2026-08-11T06:24:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2476 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T06:24:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:30Z Disabled control of system clock
I20260811 06:24:30.453867 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42217
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:42217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:24:30.793576 17988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:30.793833 17988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:30.793991 17988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:30.800695 17988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:30.800812 17988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:30.800890 17988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:30.800988 17988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:30.801055 17988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:30.810268 17988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:42217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:30.811530 17988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:30.813311 17988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:30.824841 17993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:30.824792 17997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:30.824873 17995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:30.825601 17988 server_base.cc:1034] running on GCE node
I20260811 06:24:30.826275 17988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:30.827539 17988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:30.828735 17988 hybrid_clock.cc:648] HybridClock initialized: now 1786429470828701 us; error 27 us; skew 500 ppm
I20260811 06:24:30.829057 17988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:30.831321 17988 webserver.cc:460] Webserver started at http://127.7.4.190:43967/ using document root <none> and password file <none>
I20260811 06:24:30.832250 17988 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:30.832388 17988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:30.832746 17988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:30.835625 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "ef86cb4a642a48ba8c19639fafbe54d2"
format_stamp: "Formatted at 2026-08-11 06:24:30 on dist-test-slave-07c2"
server_key: "0ab4ec4b5afb52b66078cbfc28a45fbb"
server_key_iv: "ea8b3f4d34dc7f48412e27744fe3980a"
server_key_version: "encryptionkey@0"
I20260811 06:24:30.836410 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "ef86cb4a642a48ba8c19639fafbe54d2"
format_stamp: "Formatted at 2026-08-11 06:24:30 on dist-test-slave-07c2"
server_key: "0ab4ec4b5afb52b66078cbfc28a45fbb"
server_key_iv: "ea8b3f4d34dc7f48412e27744fe3980a"
server_key_version: "encryptionkey@0"
I20260811 06:24:30.841115 17988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:24:30.844388 18004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:30.845616 17988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:30.845791 17988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "ef86cb4a642a48ba8c19639fafbe54d2"
format_stamp: "Formatted at 2026-08-11 06:24:30 on dist-test-slave-07c2"
server_key: "0ab4ec4b5afb52b66078cbfc28a45fbb"
server_key_iv: "ea8b3f4d34dc7f48412e27744fe3980a"
server_key_version: "encryptionkey@0"
I20260811 06:24:30.846009 17988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:30.851505 17988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:30.852234 17988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:30.852514 17988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:30.870112 17988 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:42217
I20260811 06:24:30.870195 18059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:42217 every 8 connection(s)
I20260811 06:24:30.871371 17988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:30.874729 18060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:30.874917 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 17988
I20260811 06:24:30.875118 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:24:30.875816 7186 external_mini_cluster.cc:1428] Setting key 209ec66170d1789c4a52e1d6028e7591
I20260811 06:24:30.883882 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:30.877732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.7.4.190:42217)
0811 06:24:30.878767 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:30.878777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:30.878813 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:24:30.880472 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:30.880491 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:30.880911 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:30.881231 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:30.882322 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:30.882328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:30.882339 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:30.882352 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:30.882412 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:30.882426 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:30.882588 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:30.882714 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:24:30.882996 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":408,"threads_started":1}
I20260811 06:24:30.887523 18060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2: Bootstrap starting.
I20260811 06:24:30.890249 18060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:30.891275 18060 log.cc:824] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:30.893802 18060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2: No bootstrap required, opened a new log
I20260811 06:24:30.900867 18060 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } }
I20260811 06:24:30.901197 18060 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:30.901330 18060 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef86cb4a642a48ba8c19639fafbe54d2, State: Initialized, Role: FOLLOWER
I20260811 06:24:30.901919 18060 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } }
I20260811 06:24:30.902221 18060 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:30.902338 18060 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:30.902454 18060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:30.904470 18060 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } }
I20260811 06:24:30.904907 18060 leader_election.cc:302] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef86cb4a642a48ba8c19639fafbe54d2; no voters:
I20260811 06:24:30.905462 18060 leader_election.cc:288] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:24:30.905627 18067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:30.906860 18067 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [term 1 LEADER]: Becoming Leader. State: Replica: ef86cb4a642a48ba8c19639fafbe54d2, State: Running, Role: LEADER
I20260811 06:24:30.907562 18060 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:30.907506 18067 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } }
I20260811 06:24:30.910701 18069 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef86cb4a642a48ba8c19639fafbe54d2. Latest consensus state: current_term: 1 leader_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } } }
I20260811 06:24:30.911358 18069 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:30.910547 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef86cb4a642a48ba8c19639fafbe54d2" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42217 } } }
I20260811 06:24:30.911949 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:30.916330 18077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:30.921381 18077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:30.932169 18077 catalog_manager.cc:1347] Generated new cluster ID: eeb44b772a6e40eb931eb158b2b991db
I20260811 06:24:30.932276 18077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:30.941833 18077 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:24:30.942253 18077 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:30.957592 18077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef86cb4a642a48ba8c19639fafbe54d2: Generated new TSK 0
I20260811 06:24:30.958460 18077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:24:30.967872 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--builtin_ntp_servers=127.7.4.148:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:24:31.297416 18088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:31.297655 18088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:31.297809 18088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:31.304800 18088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:31.304883 18088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:31.305065 18088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:31.314083 18088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:31.315214 18088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:31.317092 18088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:31.329838 18093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:31.329885 18094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:31.329913 18096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:31.330641 18088 server_base.cc:1034] running on GCE node
I20260811 06:24:31.331212 18088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:31.332096 18088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:31.333310 18088 hybrid_clock.cc:648] HybridClock initialized: now 1786429471333278 us; error 46 us; skew 500 ppm
I20260811 06:24:31.333638 18088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:31.335758 18088 webserver.cc:460] Webserver started at http://127.7.4.129:42155/ using document root <none> and password file <none>
I20260811 06:24:31.336414 18088 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:31.336544 18088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:31.336889 18088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:31.339519 18088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "aaec68fa877f4cc9861e8d150737a63a"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "0721c4f3f2523eb0f48bd55f5ece2a77"
server_key_iv: "bcd510115a1ee68583959f3f9402fe90"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.340297 18088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "aaec68fa877f4cc9861e8d150737a63a"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "0721c4f3f2523eb0f48bd55f5ece2a77"
server_key_iv: "bcd510115a1ee68583959f3f9402fe90"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.345026 18088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 06:24:31.348238 18102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:31.349540 18088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:24:31.349680 18088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "aaec68fa877f4cc9861e8d150737a63a"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "0721c4f3f2523eb0f48bd55f5ece2a77"
server_key_iv: "bcd510115a1ee68583959f3f9402fe90"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.349854 18088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:31.355206 18088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:31.356026 18088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:31.356321 18088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:31.357172 18088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:31.358451 18088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:31.358539 18088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:31.358618 18088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:31.358687 18088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:31.391986 18088 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:44039
I20260811 06:24:31.392041 18224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:44039 every 8 connection(s)
I20260811 06:24:31.393379 18088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:31.397253 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18088
I20260811 06:24:31.397444 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:24:31.398312 7186 external_mini_cluster.cc:1428] Setting key 2d0beed9d878149adea1ff7574e4005d
I20260811 06:24:31.401713 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--builtin_ntp_servers=127.7.4.148:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:24:31.402598 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:31.395938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44343 (local address 127.7.4.190:42217)
0811 06:24:31.396387 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:31.396391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:31.397668 (+ 1277us) server_negotiation.cc:407] Connection header received
0811 06:24:31.400209 (+ 2541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:31.400226 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:31.400288 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:31.400424 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:31.401882 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:31.401891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:31.401898 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:31.401924 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:31.401984 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:31.401997 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:31.402114 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:31.402342 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:24:31.402391 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260811 06:24:31.403864 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:31.396370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42217 (local address 127.7.4.129:44343)
0811 06:24:31.397504 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:31.397552 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:31.399864 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:31.400676 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:31.400687 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:31.400796 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:31.401496 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:31.401502 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:31.401554 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:31.401661 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:31.402098 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:31.402112 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:31.402122 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:24:31.402394 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 06:24:31.402626 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":324,"threads_started":1}
I20260811 06:24:31.405346 18225 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42217
I20260811 06:24:31.405670 18225 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:31.406313 18225 heartbeater.cc:511] Master 127.7.4.190:42217 requested a full tablet report, sending...
I20260811 06:24:31.408564 18021 ts_manager.cc:194] Registered new tserver with Master: aaec68fa877f4cc9861e8d150737a63a (127.7.4.129:44039)
W20260811 06:24:31.744244 18229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:31.744498 18229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:31.744680 18229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:31.751858 18229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:31.751981 18229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:31.752156 18229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:31.761087 18229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:31.762279 18229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:31.764030 18229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:31.776504 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:31.776540 18238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:31.776461 18235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:31.777439 18229 server_base.cc:1034] running on GCE node
I20260811 06:24:31.777923 18229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:31.778713 18229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:31.779902 18229 hybrid_clock.cc:648] HybridClock initialized: now 1786429471779862 us; error 37 us; skew 500 ppm
I20260811 06:24:31.780200 18229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:31.782296 18229 webserver.cc:460] Webserver started at http://127.7.4.130:42559/ using document root <none> and password file <none>
I20260811 06:24:31.782946 18229 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:31.783062 18229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:31.783504 18229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:31.786194 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "fab044ba3db841f0acabe7ab9e01bd7c"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "bed95a2bc032510d4e4e244f76be1f4e"
server_key_iv: "6e7a9bd11f3af7d9df60cba9b0eeaf06"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.787168 18229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "fab044ba3db841f0acabe7ab9e01bd7c"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "bed95a2bc032510d4e4e244f76be1f4e"
server_key_iv: "6e7a9bd11f3af7d9df60cba9b0eeaf06"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.792066 18229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 06:24:31.795392 18244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:31.796638 18229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:24:31.796808 18229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "fab044ba3db841f0acabe7ab9e01bd7c"
format_stamp: "Formatted at 2026-08-11 06:24:31 on dist-test-slave-07c2"
server_key: "bed95a2bc032510d4e4e244f76be1f4e"
server_key_iv: "6e7a9bd11f3af7d9df60cba9b0eeaf06"
server_key_version: "encryptionkey@0"
I20260811 06:24:31.797013 18229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:31.803011 18229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:31.803857 18229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:31.804128 18229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:31.805078 18229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:31.806486 18229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:31.806624 18229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:24:31.806746 18229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:31.806849 18229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:31.842684 18229 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:43057
I20260811 06:24:31.842772 18367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:43057 every 8 connection(s)
I20260811 06:24:31.844054 18229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:31.851485 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18229
I20260811 06:24:31.851667 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:24:31.852113 7186 external_mini_cluster.cc:1428] Setting key 94f37001ea187b2764640e655c943564
I20260811 06:24:31.852118 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:31.846598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:50911 (local address 127.7.4.190:42217)
0811 06:24:31.846789 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:31.846794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:31.848105 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 06:24:31.849872 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:31.849878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:31.849970 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:31.850085 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:31.851319 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:31.851325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:31.851330 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:31.851349 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:31.851388 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:31.851400 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:31.851528 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:31.851756 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:24:31.851828 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:24:31.852994 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:31.847072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42217 (local address 127.7.4.130:50911)
0811 06:24:31.847948 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:31.847989 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:31.849607 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:31.850312 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:31.850320 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:31.850402 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:31.851008 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:31.851010 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:31.851045 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:31.851123 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:31.851498 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:31.851510 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:31.851517 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:24:31.851834 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 06:24:31.852072 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":302,"threads_started":1}
I20260811 06:24:31.854861 18368 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42217
I20260811 06:24:31.855067 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--builtin_ntp_servers=127.7.4.148:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:24:31.855185 18368 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:31.855762 18368 heartbeater.cc:511] Master 127.7.4.190:42217 requested a full tablet report, sending...
I20260811 06:24:31.857200 18021 ts_manager.cc:194] Registered new tserver with Master: fab044ba3db841f0acabe7ab9e01bd7c (127.7.4.130:43057)
W20260811 06:24:32.193023 18371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:32.193269 18371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:32.193357 18371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:32.200213 18371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:32.200300 18371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:32.200449 18371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:32.209547 18371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:32.210665 18371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:32.212364 18371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:32.225368 18377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:32.225289 18378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:32.225328 18380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:32.226349 18371 server_base.cc:1034] running on GCE node
I20260811 06:24:32.226902 18371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:32.227867 18371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:32.229068 18371 hybrid_clock.cc:648] HybridClock initialized: now 1786429472229032 us; error 24 us; skew 500 ppm
I20260811 06:24:32.229383 18371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:24:32.231662 18371 webserver.cc:460] Webserver started at http://127.7.4.131:39933/ using document root <none> and password file <none>
I20260811 06:24:32.232321 18371 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:32.232439 18371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:32.232800 18371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:32.235581 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "c7d82daa21134f8bb37099875626fbed"
format_stamp: "Formatted at 2026-08-11 06:24:32 on dist-test-slave-07c2"
server_key: "12f60297b72379e70698d340c8f8028c"
server_key_iv: "ca4f20297c01fed8a85f241797abfde7"
server_key_version: "encryptionkey@0"
I20260811 06:24:32.236385 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "c7d82daa21134f8bb37099875626fbed"
format_stamp: "Formatted at 2026-08-11 06:24:32 on dist-test-slave-07c2"
server_key: "12f60297b72379e70698d340c8f8028c"
server_key_iv: "ca4f20297c01fed8a85f241797abfde7"
server_key_version: "encryptionkey@0"
I20260811 06:24:32.241218 18371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:24:32.244416 18386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:32.245644 18371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:32.245819 18371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "c7d82daa21134f8bb37099875626fbed"
format_stamp: "Formatted at 2026-08-11 06:24:32 on dist-test-slave-07c2"
server_key: "12f60297b72379e70698d340c8f8028c"
server_key_iv: "ca4f20297c01fed8a85f241797abfde7"
server_key_version: "encryptionkey@0"
I20260811 06:24:32.246037 18371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:32.251636 18371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:32.252381 18371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:32.252656 18371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:32.253579 18371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:32.254899 18371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:32.255012 18371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:32.255154 18371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:32.255250 18371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:32.288252 18371 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:34215
I20260811 06:24:32.288347 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:34215 every 8 connection(s)
I20260811 06:24:32.289573 18371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:32.294261 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18371
I20260811 06:24:32.294430 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:24:32.294900 7186 external_mini_cluster.cc:1428] Setting key 38dc28bd9d0953cd2cb2f96ae2d228a6
I20260811 06:24:32.298162 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:32.292121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:58597 (local address 127.7.4.190:42217)
0811 06:24:32.292319 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:32.292324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:32.293633 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 06:24:32.295694 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:32.295701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:32.295783 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:32.295896 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:32.297405 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:32.297410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:32.297413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:32.297428 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:32.297460 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:32.297472 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:32.297567 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:32.297836 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:24:32.297902 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:24:32.299034 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:32.292618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42217 (local address 127.7.4.131:58597)
0811 06:24:32.293490 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:32.293525 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:32.295343 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:32.296161 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:32.296198 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:32.296316 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:32.297069 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:24:32.297075 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:32.297127 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:24:32.297239 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:32.297576 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:32.297588 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:32.297596 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:24:32.297893 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 06:24:32.298140 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":266,"threads_started":1}
I20260811 06:24:32.300309 18510 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42217
I20260811 06:24:32.300649 18510 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:32.301265 18510 heartbeater.cc:511] Master 127.7.4.190:42217 requested a full tablet report, sending...
I20260811 06:24:32.302561 18021 ts_manager.cc:194] Registered new tserver with Master: c7d82daa21134f8bb37099875626fbed (127.7.4.131:34215)
I20260811 06:24:32.310053 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:24:32.316076 18227 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:24:32.314508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.7.4.190:42217)
0811 06:24:32.314656 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:32.314660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:32.315162 (+ 502us) server_negotiation.cc:407] Connection header received
0811 06:24:32.315384 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:32.315389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:32.315437 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:32.315534 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:32.315879 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54396: BlockingRecv error: recv got EOF from 127.0.0.1:54396 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:24:32.317050 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18088
I20260811 06:24:32.320530 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:32.318123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.7.4.129:44039)
0811 06:24:32.318648 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:32.318656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:32.318680 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:24:32.318736 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:32.318741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:32.318909 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:32.319025 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:32.319898 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:32.319903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:32.319913 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:32.319925 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:32.319962 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:32.319976 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:32.320077 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:32.320125 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 06:24:32.320237 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":246,"threads_started":1}
I20260811 06:24:32.321672 18220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:32.403052 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18088
I20260811 06:24:32.426642 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18229
I20260811 06:24:32.430634 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:32.427692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.7.4.130:43057)
0811 06:24:32.428458 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:32.428470 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:32.428510 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:24:32.428615 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:32.428628 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:32.428939 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:32.429081 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:32.429966 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:32.429972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:32.429982 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:32.430002 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:32.430045 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:32.430059 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:32.430171 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:32.430218 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 06:24:32.430330 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":372,"threads_started":1}
I20260811 06:24:32.431923 18363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:32.513783 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18229
I20260811 06:24:32.534917 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18371
I20260811 06:24:32.540017 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:32.536429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.7.4.131:34215)
0811 06:24:32.537355 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:32.537364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:32.537405 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:24:32.537488 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:32.537494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:32.537723 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:32.537977 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:32.539176 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:32.539182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:32.539193 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:24:32.539210 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:32.539251 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:24:32.539265 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:32.539478 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:32.539536 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 06:24:32.539646 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":476,"threads_started":1}
I20260811 06:24:32.541314 18505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:32.624118 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18371
I20260811 06:24:32.645423 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 17988
I20260811 06:24:32.646653 18054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:32.749888 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 17988
2026-08-11T06:24:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2382 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:24:32 dist-test-slave-07c2 krb5kdc[18528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:24:32 dist-test-slave-07c2 krb5kdc[18528](info): set up 2 sockets
Aug 11 06:24:32 dist-test-slave-07c2 krb5kdc[18528](info): commencing operation
krb5kdc: starting...
W20260811 06:24:33.918783 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:24:34 dist-test-slave-07c2 krb5kdc[18528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:24:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:24:34.180578 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46709
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:44893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:34.503966 18541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:34.504242 18541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:34.504510 18541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:34.511687 18541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:24:34.511808 18541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:34.511890 18541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:34.511994 18541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:24:34.512079 18541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:24:34.521531 18541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:44893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:34.522758 18541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:34.524075 18541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:34.535231 18547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:34.535450 18548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:34.535230 18550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:34.536330 18541 server_base.cc:1034] running on GCE node
I20260811 06:24:34.537068 18541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:34.538342 18541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:34.539602 18541 hybrid_clock.cc:648] HybridClock initialized: now 1786429474539555 us; error 26 us; skew 500 ppm
I20260811 06:24:34.539963 18541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:34 dist-test-slave-07c2 krb5kdc[18528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:34.546773 18541 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:24:34.548301 18541 webserver.cc:460] Webserver started at http://127.7.4.190:36199/ using document root <none> and password file <none>
I20260811 06:24:34.549021 18541 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:34.549161 18541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:34.549487 18541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:34.552461 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "c105d5ac4065417497f0a2596842639a"
format_stamp: "Formatted at 2026-08-11 06:24:34 on dist-test-slave-07c2"
server_key: "637d6760f7c90b344be6ca2e13db7ff7"
server_key_iv: "7e76a860b2b566a774fc4cb4b8dd82b6"
server_key_version: "encryptionkey@0"
I20260811 06:24:34.553193 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "c105d5ac4065417497f0a2596842639a"
format_stamp: "Formatted at 2026-08-11 06:24:34 on dist-test-slave-07c2"
server_key: "637d6760f7c90b344be6ca2e13db7ff7"
server_key_iv: "7e76a860b2b566a774fc4cb4b8dd82b6"
server_key_version: "encryptionkey@0"
I20260811 06:24:34.558159 18541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:24:34.561440 18558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:34.562640 18541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:24:34.562812 18541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "c105d5ac4065417497f0a2596842639a"
format_stamp: "Formatted at 2026-08-11 06:24:34 on dist-test-slave-07c2"
server_key: "637d6760f7c90b344be6ca2e13db7ff7"
server_key_iv: "7e76a860b2b566a774fc4cb4b8dd82b6"
server_key_version: "encryptionkey@0"
I20260811 06:24:34.562990 18541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:34.593848 18541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:34.597985 18541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:34.598256 18541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:34.617110 18541 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:46709
I20260811 06:24:34.617190 18615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:46709 every 8 connection(s)
I20260811 06:24:34.618438 18541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:24:34.622130 18616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:34.624914 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18541
I20260811 06:24:34.625142 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:24:34.625953 7186 external_mini_cluster.cc:1428] Setting key 49574d4adde3211e61cce00439f155dd
I20260811 06:24:34.634562 18616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: Bootstrap starting.
I20260811 06:24:34.638124 18616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:34.639412 18616 log.cc:824] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:24:34 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:34.642330 18616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: No bootstrap required, opened a new log
I20260811 06:24:34.650166 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:34.628068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.7.4.190:46709)
0811 06:24:34.628860 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:34.628871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:34.628908 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:24:34.630504 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:34.630522 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:34.630870 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:34.631180 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:34.632998 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:34.634296 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:34.636068 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:34.636675 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:34.636789 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:34.641756 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:34.641763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:34.641776 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:34.641810 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:34.645758 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:34.646442 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:34.646452 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:34.646464 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:34.646668 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:34.647047 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:34.647052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:34.647056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:34.647803 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:34.648152 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:34.648440 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 06:24:34.648876 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":419,"threads_started":1}
I20260811 06:24:34.652736 18616 raft_consensus.cc:348] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } }
I20260811 06:24:34.653075 18616 raft_consensus.cc:374] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:34.653198 18616 raft_consensus.cc:729] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c105d5ac4065417497f0a2596842639a, State: Initialized, Role: FOLLOWER
I20260811 06:24:34.653724 18616 consensus_queue.cc:260] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } }
I20260811 06:24:34.653946 18616 raft_consensus.cc:388] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:24:34.654073 18616 raft_consensus.cc:482] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:24:34.654225 18616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:34.656497 18616 raft_consensus.cc:504] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } }
I20260811 06:24:34.656965 18616 leader_election.cc:302] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c105d5ac4065417497f0a2596842639a; no voters:
I20260811 06:24:34.657537 18616 leader_election.cc:288] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:24:34.657701 18621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:34.658785 18621 raft_consensus.cc:686] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [term 1 LEADER]: Becoming Leader. State: Replica: c105d5ac4065417497f0a2596842639a, State: Running, Role: LEADER
I20260811 06:24:34.659332 18621 consensus_queue.cc:237] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } }
I20260811 06:24:34.659897 18616 sys_catalog.cc:564] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:24:34.662326 18623 sys_catalog.cc:455] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c105d5ac4065417497f0a2596842639a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } } }
I20260811 06:24:34.662358 18624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c105d5ac4065417497f0a2596842639a. Latest consensus state: current_term: 1 leader_uuid: "c105d5ac4065417497f0a2596842639a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c105d5ac4065417497f0a2596842639a" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46709 } } }
I20260811 06:24:34.662750 18624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:34.662750 18623 sys_catalog.cc:458] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a [sys.catalog]: This master's current role is: LEADER
I20260811 06:24:34.666277 18630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:24:34.671329 18630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:24:34.679605 18630 catalog_manager.cc:1347] Generated new cluster ID: 2ba2bcffceaa4d5a8dba7dbedcfe4c05
I20260811 06:24:34.679749 18630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:24:34.700354 18630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:24:34.702056 18630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:24:34.709074 18630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: Generated new TSK 0
I20260811 06:24:34.709973 18630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:24:34.782752 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:44893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:35.117344 18644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:35.117605 18644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:35.117731 18644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:35.124867 18644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:35.124981 18644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:35.125149 18644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:24:35.134698 18644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:44893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:35.135958 18644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:35.137236 18644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:35.150208 18650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:35.150403 18653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:35.151010 18644 server_base.cc:1034] running on GCE node
W20260811 06:24:35.151046 18651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:35.151959 18644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:35.152971 18644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:35.154233 18644 hybrid_clock.cc:648] HybridClock initialized: now 1786429475154192 us; error 52 us; skew 500 ppm
I20260811 06:24:35.154546 18644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:35 dist-test-slave-07c2 krb5kdc[18528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:35.157595 18644 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:24:35.159018 18644 webserver.cc:460] Webserver started at http://127.7.4.129:44421/ using document root <none> and password file <none>
I20260811 06:24:35.159726 18644 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:35.159871 18644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:35.160176 18644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:35.162930 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "95067b7a7fa141278aa4e0c0e59c0a7f"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "3b977b809c9a1a903d18431e377a8adf"
server_key_iv: "d7a0cd327f9767ae5bc72dc64976faf3"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.163693 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "95067b7a7fa141278aa4e0c0e59c0a7f"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "3b977b809c9a1a903d18431e377a8adf"
server_key_iv: "d7a0cd327f9767ae5bc72dc64976faf3"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.168851 18644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:24:35.172078 18660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.173314 18644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:24:35.173501 18644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "95067b7a7fa141278aa4e0c0e59c0a7f"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "3b977b809c9a1a903d18431e377a8adf"
server_key_iv: "d7a0cd327f9767ae5bc72dc64976faf3"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.173705 18644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:35.192602 18644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:35.196262 18644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:35.196539 18644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:35.197429 18644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:35.198662 18644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:35.198761 18644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.198915 18644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:35.198984 18644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.232766 18644 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:33629
I20260811 06:24:35.232851 18787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:33629 every 8 connection(s)
I20260811 06:24:35.234004 18644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:24:35.240109 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18644
I20260811 06:24:35.240320 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:24:35.240995 7186 external_mini_cluster.cc:1428] Setting key 11bd51aab6b030ba173269341d50a0f5
Aug 11 06:24:35 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:35.255415 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:35.236542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:47847 (local address 127.7.4.190:46709)
0811 06:24:35.236985 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:35.236989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:35.237931 (+ 942us) server_negotiation.cc:407] Connection header received
0811 06:24:35.239836 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:35.239841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:35.239930 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:35.240054 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:35.241648 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.242537 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.244259 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.244990 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.245151 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:35.248510 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:35.248519 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:35.248526 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:35.248586 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:35.252169 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:35.252870 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:35.252878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:35.252885 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:35.253040 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:35.253445 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:35.253451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:35.253457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:35.253844 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:35.253998 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:35.254638 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 06:24:35.254912 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":217,"threads_started":1}
I20260811 06:24:35.256002 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:35.237060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46709 (local address 127.7.4.129:47847)
0811 06:24:35.237776 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:35.237812 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:35.239564 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:35.240338 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:35.240346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:35.240885 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:35.241471 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:35.241489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.242739 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.242745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:35.244097 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:35.244110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.245221 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.245225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:35.245407 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:35.246071 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:35.246098 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:35.248309 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:35.252315 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:35.252321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:35.252332 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:35.252738 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:35.253149 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:35.253152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:35.253154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:35.253329 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:35.253964 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:35.253969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:35.254271 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 06:24:35.254484 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 06:24:35.254836 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":277,"threads_started":1}
I20260811 06:24:35.257589 18788 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46709
I20260811 06:24:35.257877 18788 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:35.258580 18788 heartbeater.cc:511] Master 127.7.4.190:46709 requested a full tablet report, sending...
I20260811 06:24:35.260726 18577 ts_manager.cc:194] Registered new tserver with Master: 95067b7a7fa141278aa4e0c0e59c0a7f (127.7.4.129:33629)
I20260811 06:24:35.262171 18577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:47847
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:24:35.303511 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:44893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:35.634045 18794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:35.634289 18794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:35.634478 18794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:35.641445 18794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:35.641537 18794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:35.641744 18794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:24:35.650959 18794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:44893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:35.652103 18794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:35.653391 18794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:35.666327 18800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:35.666285 18803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:35.666383 18801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:35.668598 18794 server_base.cc:1034] running on GCE node
I20260811 06:24:35.669188 18794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:35.670059 18794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:35.671331 18794 hybrid_clock.cc:648] HybridClock initialized: now 1786429475671257 us; error 45 us; skew 500 ppm
I20260811 06:24:35.671633 18794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:35 dist-test-slave-07c2 krb5kdc[18528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:35.674772 18794 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:24:35.676172 18794 webserver.cc:460] Webserver started at http://127.7.4.130:38899/ using document root <none> and password file <none>
I20260811 06:24:35.676772 18794 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:35.676889 18794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:35.677139 18794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:35.679952 18794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "dcf3a443bce4402ea73a682e9cefd3eb"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "69c51e2419c8ca002aca1281be5afbec"
server_key_iv: "23ca81b020892a1f9bfb09124d358f61"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.680644 18794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "dcf3a443bce4402ea73a682e9cefd3eb"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "69c51e2419c8ca002aca1281be5afbec"
server_key_iv: "23ca81b020892a1f9bfb09124d358f61"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.685411 18794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 06:24:35.688632 18810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.689888 18794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:24:35.690019 18794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "dcf3a443bce4402ea73a682e9cefd3eb"
format_stamp: "Formatted at 2026-08-11 06:24:35 on dist-test-slave-07c2"
server_key: "69c51e2419c8ca002aca1281be5afbec"
server_key_iv: "23ca81b020892a1f9bfb09124d358f61"
server_key_version: "encryptionkey@0"
I20260811 06:24:35.690176 18794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:35.709604 18794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:35.713826 18794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:35.714059 18794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:35.715036 18794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:35.716364 18794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:35.716441 18794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.716536 18794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:35.716579 18794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:35.750394 18794 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:46879
I20260811 06:24:35.750451 18937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:46879 every 8 connection(s)
I20260811 06:24:35.751737 18794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:24:35.759979 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18794
I20260811 06:24:35.760154 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:24:35.760715 7186 external_mini_cluster.cc:1428] Setting key 43ef340e33e2e02a00e038ab9470d1c6
Aug 11 06:24:35 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:35.773150 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:35.754474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:42543 (local address 127.7.4.190:46709)
0811 06:24:35.754666 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:35.754671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:35.755891 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 06:24:35.757722 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:35.757728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:35.757818 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:35.757932 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:35.759441 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.760567 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.762425 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.763041 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.763186 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:35.767254 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:35.767258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:35.767261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:35.767343 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:35.769939 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:35.770841 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:35.770844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:35.770846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:35.770970 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:35.771503 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:35.771506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:35.771508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:35.771781 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:35.771892 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:35.772749 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 06:24:35.772912 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:24:35.774616 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:35.754950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46709 (local address 127.7.4.130:42543)
0811 06:24:35.755737 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:35.755776 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:35.757454 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:35.758167 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:35.758175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:35.758680 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:35.759224 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:35.759242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.760706 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.760710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:35.762268 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:35.762282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:35.763261 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:35.763266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:35.763516 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:35.764216 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:35.764242 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:35.767045 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:35.770079 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:35.770091 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:35.770107 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:35.770726 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:35.771106 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:35.771115 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:35.771121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:35.771391 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:35.771910 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:35.771921 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:35.772363 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 06:24:35.772662 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:24:35.773228 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":348,"threads_started":1}
I20260811 06:24:35.776026 18938 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46709
I20260811 06:24:35.776329 18938 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:35.777139 18938 heartbeater.cc:511] Master 127.7.4.190:46709 requested a full tablet report, sending...
I20260811 06:24:35.778762 18577 ts_manager.cc:194] Registered new tserver with Master: dcf3a443bce4402ea73a682e9cefd3eb (127.7.4.130:46879)
I20260811 06:24:35.779629 18577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:42543
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:24:35.826942 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:44893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:24:36.155809 18943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:24:36.156083 18943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:24:36.156266 18943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:24:36.163522 18943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:24:36.163631 18943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:24:36.163806 18943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:24:36.173257 18943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:44893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:24:36.174386 18943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:24:36.175670 18943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:24:36.188657 18949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:36.188763 18951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:24:36.188777 18948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:24:36.189551 18943 server_base.cc:1034] running on GCE node
I20260811 06:24:36.190126 18943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:24:36.191020 18943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:24:36.192251 18943 hybrid_clock.cc:648] HybridClock initialized: now 1786429476192214 us; error 48 us; skew 500 ppm
I20260811 06:24:36.192615 18943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:24:36 dist-test-slave-07c2 krb5kdc[18528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429476, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:24:36.196041 18943 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:24:36.197407 18943 webserver.cc:460] Webserver started at http://127.7.4.131:37815/ using document root <none> and password file <none>
I20260811 06:24:36.198033 18943 fs_manager.cc:359] Metadata directory not provided
I20260811 06:24:36.198184 18943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:24:36.198478 18943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:24:36.201265 18943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "1b30839c1ddb44eaab6f61638d6a3da5"
format_stamp: "Formatted at 2026-08-11 06:24:36 on dist-test-slave-07c2"
server_key: "badd34bd524877a2321520ef73679b59"
server_key_iv: "b4fcbf14639a83b21a7c3edba3b404d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:36.202029 18943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "1b30839c1ddb44eaab6f61638d6a3da5"
format_stamp: "Formatted at 2026-08-11 06:24:36 on dist-test-slave-07c2"
server_key: "badd34bd524877a2321520ef73679b59"
server_key_iv: "b4fcbf14639a83b21a7c3edba3b404d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:36.206887 18943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 06:24:36.210228 18958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:36.211483 18943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:24:36.211644 18943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "1b30839c1ddb44eaab6f61638d6a3da5"
format_stamp: "Formatted at 2026-08-11 06:24:36 on dist-test-slave-07c2"
server_key: "badd34bd524877a2321520ef73679b59"
server_key_iv: "b4fcbf14639a83b21a7c3edba3b404d5"
server_key_version: "encryptionkey@0"
I20260811 06:24:36.211848 18943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:24:36.234396 18943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:24:36.238471 18943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:24:36.238762 18943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:24:36.239777 18943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:24:36.241271 18943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:24:36.241374 18943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:36.241495 18943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:24:36.241576 18943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:24:36.265139 18788 heartbeater.cc:503] Master 127.7.4.190:46709 was elected leader, sending a full tablet report...
I20260811 06:24:36.275658 18943 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:34013
I20260811 06:24:36.275676 19082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:34013 every 8 connection(s)
I20260811 06:24:36.276930 18943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:24:36.284106 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 18943
I20260811 06:24:36.284286 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:24:36.284722 7186 external_mini_cluster.cc:1428] Setting key 90f71e9778625d88183f0ac5594db173
Aug 11 06:24:36 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429476, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:24:36.296445 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.279391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:48491 (local address 127.7.4.190:46709)
0811 06:24:36.279884 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.279888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.280670 (+ 782us) server_negotiation.cc:407] Connection header received
0811 06:24:36.282467 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.282473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.282561 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.282681 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:36.284187 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.285094 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.286670 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.287102 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.287232 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.290585 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:36.290590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:36.290593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:36.290622 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:36.293377 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.294153 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.294165 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.294167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.294285 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.294708 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.294717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.294719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.294964 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:36.295121 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.295877 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.296083 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":297,"threads_started":1}
I20260811 06:24:36.297286 19086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.279848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46709 (local address 127.7.4.131:48491)
0811 06:24:36.280525 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.280568 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.282173 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.282911 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.282919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.283489 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:24:36.284021 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.284036 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.285260 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.285264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.286524 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.286536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.287367 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.287378 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.287598 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.288252 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:24:36.288273 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:24:36.290392 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:24:36.293517 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:36.293525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:36.293537 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:36.294007 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:36.294395 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:24:36.294400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:24:36.294403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:24:36.294575 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:24:36.295100 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:24:36.295107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:24:36.295495 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.295772 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.296235 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":277,"threads_started":1}
I20260811 06:24:36.298605 19083 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46709
I20260811 06:24:36.298893 19083 heartbeater.cc:464] Registering TS with master...
I20260811 06:24:36.299703 19083 heartbeater.cc:511] Master 127.7.4.190:46709 requested a full tablet report, sending...
I20260811 06:24:36.301079 18577 ts_manager.cc:194] Registered new tserver with Master: 1b30839c1ddb44eaab6f61638d6a3da5 (127.7.4.131:34013)
I20260811 06:24:36.301815 18577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:48491
I20260811 06:24:36.311950 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:24:36.328007 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.316739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46502 (local address 127.7.4.190:46709)
0811 06:24:36.316882 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.316886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.317391 (+ 505us) server_negotiation.cc:407] Connection header received
0811 06:24:36.317514 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.317517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.317571 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.317650 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:36.318990 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.319706 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.320877 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.321294 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.321385 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.323547 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:36.323551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:36.323554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:36.323581 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:36.325798 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.326418 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.326421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.326424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.326531 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.326832 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.326835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.326837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.327116 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:36.327323 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.327659 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.327806 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:24:36.331835 18577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:24:36.338792 18577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:24:36.363672 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.351040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.7.4.130:46879)
0811 06:24:36.352566 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.352579 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.354762 (+ 2183us) server_negotiation.cc:407] Connection header received
0811 06:24:36.354889 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.354896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.355144 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.355392 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:36.356582 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.358208 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.360913 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.362431 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.362579 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.362813 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.363109 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.363314 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":1144,"threads_started":1}
I20260811 06:24:36.363785 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.350387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:46879 (local address 127.0.0.1:55808)
0811 06:24:36.354292 (+ 3905us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.354316 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.354536 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.355686 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.355691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.355754 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:36.356380 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.356393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.358351 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.358356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.360734 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.360748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.362641 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.362647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.362851 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.362908 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.363062 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.363246 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3730,"mutex_wait_us":32,"thread_start_us":2351,"threads_started":1}
I20260811 06:24:36.363839 19100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.350190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52944)
0811 06:24:36.353231 (+ 3041us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.353295 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.354603 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.356171 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.356175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.356198 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:36.356662 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.356675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.358640 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.358649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.360896 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.360911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.362973 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.362977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.363164 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.363196 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.363315 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.363449 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2741,"thread_start_us":3729,"threads_started":1}
I20260811 06:24:36.364557 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.350506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52944 (local address 127.7.4.129:33629)
0811 06:24:36.355276 (+ 4770us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.355306 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.355336 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:24:36.355429 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.355435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.355656 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.355885 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:36.356865 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.358466 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.361076 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.362490 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.362593 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.362837 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.363993 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.364157 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4563,"thread_start_us":4361,"threads_started":1}
I20260811 06:24:36.365170 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.355545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:34013 (local address 127.0.0.1:44646)
0811 06:24:36.356211 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.356233 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.356381 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.357008 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.357013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.357353 (+ 340us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:36.357798 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.357813 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.359859 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.359865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.362087 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.362101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.364438 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.364445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.364571 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.364594 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.364694 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.364806 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":2,"thread_start_us":348,"threads_started":1}
I20260811 06:24:36.365428 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.354654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.7.4.131:34013)
0811 06:24:36.355591 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.355600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.356351 (+ 751us) server_negotiation.cc:407] Connection header received
0811 06:24:36.356560 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.356570 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.356793 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.357017 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:36.358008 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.359688 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.362286 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.363709 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.363860 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.364415 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.364901 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.365069 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":560,"threads_started":1}
I20260811 06:24:36.367992 18711 tablet_service.cc:1467] Processing CreateTablet for tablet 46891fa0e1524ec38941ac592147ec7c (DEFAULT_TABLE table=test-table [id=d9531874d51f4ceab6664564f5d39d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:36.367993 18867 tablet_service.cc:1467] Processing CreateTablet for tablet 46891fa0e1524ec38941ac592147ec7c (DEFAULT_TABLE table=test-table [id=d9531874d51f4ceab6664564f5d39d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:36.368813 19011 tablet_service.cc:1467] Processing CreateTablet for tablet 46891fa0e1524ec38941ac592147ec7c (DEFAULT_TABLE table=test-table [id=d9531874d51f4ceab6664564f5d39d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:24:36.369470 18867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46891fa0e1524ec38941ac592147ec7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:36.369468 18711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46891fa0e1524ec38941ac592147ec7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:36.369943 19011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46891fa0e1524ec38941ac592147ec7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:24:36.381199 19106 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Bootstrap starting.
I20260811 06:24:36.383980 19107 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Bootstrap starting.
I20260811 06:24:36.385457 19106 tablet_bootstrap.cc:654] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:36.386752 19106 log.cc:824] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:36.387439 19107 tablet_bootstrap.cc:654] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:36.388648 19107 log.cc:824] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:36.389676 19109 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Bootstrap starting.
I20260811 06:24:36.390451 19106 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: No bootstrap required, opened a new log
I20260811 06:24:36.390763 19106 ts_tablet_manager.cc:1397] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 06:24:36.391812 19107 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: No bootstrap required, opened a new log
I20260811 06:24:36.392077 19107 ts_tablet_manager.cc:1397] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:24:36.393594 19109 tablet_bootstrap.cc:654] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Neither blocks nor log segments found. Creating new log.
I20260811 06:24:36.394802 19109 log.cc:824] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:24:36.397765 19109 tablet_bootstrap.cc:492] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: No bootstrap required, opened a new log
I20260811 06:24:36.398025 19109 ts_tablet_manager.cc:1397] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:24:36.400410 19107 raft_consensus.cc:348] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.400751 19107 raft_consensus.cc:374] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:36.400873 19107 raft_consensus.cc:729] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95067b7a7fa141278aa4e0c0e59c0a7f, State: Initialized, Role: FOLLOWER
I20260811 06:24:36.401226 19106 raft_consensus.cc:348] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.401624 19106 raft_consensus.cc:374] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:36.401472 19107 consensus_queue.cc:260] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.401721 19106 raft_consensus.cc:729] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b30839c1ddb44eaab6f61638d6a3da5, State: Initialized, Role: FOLLOWER
I20260811 06:24:36.403033 19107 ts_tablet_manager.cc:1428] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 06:24:36.402323 19106 consensus_queue.cc:260] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.409507 19109 raft_consensus.cc:348] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.410070 19109 raft_consensus.cc:374] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:24:36.410184 19109 raft_consensus.cc:729] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf3a443bce4402ea73a682e9cefd3eb, State: Initialized, Role: FOLLOWER
I20260811 06:24:36.410790 19083 heartbeater.cc:503] Master 127.7.4.190:46709 was elected leader, sending a full tablet report...
I20260811 06:24:36.410780 19109 consensus_queue.cc:260] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.411543 19106 ts_tablet_manager.cc:1428] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Time spent starting tablet: real 0.021s user 0.014s sys 0.007s
I20260811 06:24:36.412426 18938 heartbeater.cc:503] Master 127.7.4.190:46709 was elected leader, sending a full tablet report...
I20260811 06:24:36.414196 19109 ts_tablet_manager.cc:1428] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260811 06:24:36.486438 18789 tablet.cc:2369] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:24:36.503793 18939 tablet.cc:2369] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:24:36.507447 19115 raft_consensus.cc:482] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:24:36.507725 19115 raft_consensus.cc:504] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.509351 19115 leader_election.cc:288] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95067b7a7fa141278aa4e0c0e59c0a7f (127.7.4.129:33629), 1b30839c1ddb44eaab6f61638d6a3da5 (127.7.4.131:34013)
I20260811 06:24:36.516753 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.509771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:32791 (local address 127.7.4.129:33629)
0811 06:24:36.509970 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.509975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.510545 (+ 570us) server_negotiation.cc:407] Connection header received
0811 06:24:36.510756 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.510761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.510846 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.510946 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:36.511745 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.512550 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.514760 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.515639 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.515821 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.515930 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.516382 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.516503 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:24:36.516861 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.509699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.7.4.130:32791)
0811 06:24:36.510435 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.510454 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.510588 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.510996 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.511002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.511034 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:36.511539 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.511557 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.512693 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.512702 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.514591 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.514609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.515884 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.515890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.516071 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.516151 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.516343 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.516470 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":52,"thread_start_us":391,"threads_started":1}
I20260811 06:24:36.517053 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.510704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:36545 (local address 127.7.4.131:34013)
0811 06:24:36.510896 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.510903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.511324 (+ 421us) server_negotiation.cc:407] Connection header received
0811 06:24:36.511539 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.511544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.511602 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.511703 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:24:36.512234 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.512980 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.514972 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.515935 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.516083 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.516221 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.516686 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.516808 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:24:36.517468 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.510538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:34013 (local address 127.7.4.130:36545)
0811 06:24:36.511184 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.511200 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:24:36.511370 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:24:36.511717 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:24:36.511722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:24:36.511744 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:24:36.512077 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.512085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.513142 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.513152 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.514798 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:24:36.514810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.516195 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.516204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:24:36.516442 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.516479 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:24:36.516649 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 06:24:36.516776 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":1,"thread_start_us":335,"threads_started":1}
I20260811 06:24:36.518175 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46891fa0e1524ec38941ac592147ec7c" candidate_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" is_pre_election: true
I20260811 06:24:36.518354 19031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46891fa0e1524ec38941ac592147ec7c" candidate_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" is_pre_election: true
I20260811 06:24:36.518690 18735 raft_consensus.cc:2454] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcf3a443bce4402ea73a682e9cefd3eb in term 0.
I20260811 06:24:36.518884 19031 raft_consensus.cc:2454] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcf3a443bce4402ea73a682e9cefd3eb in term 0.
I20260811 06:24:36.519389 18811 leader_election.cc:302] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95067b7a7fa141278aa4e0c0e59c0a7f, dcf3a443bce4402ea73a682e9cefd3eb; no voters:
I20260811 06:24:36.519764 19115 raft_consensus.cc:2781] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:24:36.519910 19115 raft_consensus.cc:482] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:24:36.520021 19115 raft_consensus.cc:3037] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:36.522260 19115 raft_consensus.cc:504] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
I20260811 06:24:36.522797 19115 leader_election.cc:288] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [CANDIDATE]: Term 1 election: Requested vote from peers 95067b7a7fa141278aa4e0c0e59c0a7f (127.7.4.129:33629), 1b30839c1ddb44eaab6f61638d6a3da5 (127.7.4.131:34013)
I20260811 06:24:36.523211 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46891fa0e1524ec38941ac592147ec7c" candidate_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f"
I20260811 06:24:36.523350 19031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46891fa0e1524ec38941ac592147ec7c" candidate_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b30839c1ddb44eaab6f61638d6a3da5"
I20260811 06:24:36.523468 18735 raft_consensus.cc:3037] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:36.523538 19031 raft_consensus.cc:3037] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:24:36.526650 19031 raft_consensus.cc:2454] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcf3a443bce4402ea73a682e9cefd3eb in term 1.
I20260811 06:24:36.526650 18735 raft_consensus.cc:2454] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcf3a443bce4402ea73a682e9cefd3eb in term 1.
I20260811 06:24:36.527002 18811 leader_election.cc:302] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95067b7a7fa141278aa4e0c0e59c0a7f, dcf3a443bce4402ea73a682e9cefd3eb; no voters:
I20260811 06:24:36.527261 19115 raft_consensus.cc:2781] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:24:36.527729 19115 raft_consensus.cc:686] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 1 LEADER]: Becoming Leader. State: Replica: dcf3a443bce4402ea73a682e9cefd3eb, State: Running, Role: LEADER
I20260811 06:24:36.528218 19115 consensus_queue.cc:237] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } }
W20260811 06:24:36.528509 19084 tablet.cc:2369] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:24:36.532711 18576 catalog_manager.cc:5495] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb reported cstate change: term changed from 0 to 1, leader changed from <none> to dcf3a443bce4402ea73a682e9cefd3eb (127.7.4.130). New cstate: current_term: 1 leader_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcf3a443bce4402ea73a682e9cefd3eb" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 46879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 } health_report { overall_health: UNKNOWN } } }
I20260811 06:24:36.549134 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.544183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.7.4.130:46879)
0811 06:24:36.544399 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.544406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.544510 (+ 104us) server_negotiation.cc:407] Connection header received
0811 06:24:36.544673 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.544677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.544767 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.544882 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:24:36.545198 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.545881 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.547094 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.547525 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.547617 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.548046 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:24:36.548562 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:24:36.548682 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.548778 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.548838 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:24:36.557948 19031 raft_consensus.cc:1261] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 1 FOLLOWER]: Refusing update from remote peer dcf3a443bce4402ea73a682e9cefd3eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:36.557969 18735 raft_consensus.cc:1261] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 1 FOLLOWER]: Refusing update from remote peer dcf3a443bce4402ea73a682e9cefd3eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:24:36.558893 19115 consensus_queue.cc:1035] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b30839c1ddb44eaab6f61638d6a3da5" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:36.559253 19118 consensus_queue.cc:1035] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "95067b7a7fa141278aa4e0c0e59c0a7f" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:24:36.584609 18577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:36.584931 18577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:24:36.588641 18577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c105d5ac4065417497f0a2596842639a: Sending DeleteTablet for 3 replicas of tablet 46891fa0e1524ec38941ac592147ec7c
I20260811 06:24:36.589787 18867 tablet_service.cc:1514] Processing DeleteTablet for tablet 46891fa0e1524ec38941ac592147ec7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:36 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:55808
I20260811 06:24:36.589843 18711 tablet_service.cc:1514] Processing DeleteTablet for tablet 46891fa0e1524ec38941ac592147ec7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:36 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:52944
I20260811 06:24:36.590097 19011 tablet_service.cc:1514] Processing DeleteTablet for tablet 46891fa0e1524ec38941ac592147ec7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:24:36 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:44646
I20260811 06:24:36.592051 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18644
I20260811 06:24:36.606292 19135 tablet_replica.cc:330] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: stopping tablet replica
I20260811 06:24:36.606740 19135 raft_consensus.cc:2229] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:36.607111 19134 tablet_replica.cc:330] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: stopping tablet replica
I20260811 06:24:36.607211 19135 raft_consensus.cc:2258] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:36.607542 19134 raft_consensus.cc:2229] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:24:36.607991 19134 raft_consensus.cc:2258] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:36.608785 19133 tablet_replica.cc:330] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: stopping tablet replica
I20260811 06:24:36.609179 19133 raft_consensus.cc:2229] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:24:36.609804 19133 raft_consensus.cc:2258] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:24:36.610917 19135 ts_tablet_manager.cc:1905] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:36.611932 19133 ts_tablet_manager.cc:1905] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:24:36 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:24:36.618541 19135 ts_tablet_manager.cc:1918] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:36.618707 19135 log.cc:1196] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/46891fa0e1524ec38941ac592147ec7c
I20260811 06:24:36.619197 19135 ts_tablet_manager.cc:1939] T 46891fa0e1524ec38941ac592147ec7c P 1b30839c1ddb44eaab6f61638d6a3da5: Deleting consensus metadata
I20260811 06:24:36.620608 19133 ts_tablet_manager.cc:1918] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:24:36.620898 18559 catalog_manager.cc:4841] TS 1b30839c1ddb44eaab6f61638d6a3da5 (127.7.4.131:34013): tablet 46891fa0e1524ec38941ac592147ec7c (table test-table [id=d9531874d51f4ceab6664564f5d39d27]) successfully deleted
I20260811 06:24:36.621039 19133 log.cc:1196] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/46891fa0e1524ec38941ac592147ec7c
I20260811 06:24:36.621604 19133 ts_tablet_manager.cc:1939] T 46891fa0e1524ec38941ac592147ec7c P dcf3a443bce4402ea73a682e9cefd3eb: Deleting consensus metadata
I20260811 06:24:36.623241 18561 catalog_manager.cc:4841] TS dcf3a443bce4402ea73a682e9cefd3eb (127.7.4.130:46879): tablet 46891fa0e1524ec38941ac592147ec7c (table test-table [id=d9531874d51f4ceab6664564f5d39d27]) successfully deleted
I20260811 06:24:36.624346 19134 ts_tablet_manager.cc:1905] T 46891fa0e1524ec38941ac592147ec7c P 95067b7a7fa141278aa4e0c0e59c0a7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:24:36.626332 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.593586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.7.4.129:33629)
0811 06:24:36.593804 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.593812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.606500 (+ 12688us) server_negotiation.cc:407] Connection header received
0811 06:24:36.606705 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.606712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.606804 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.606912 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:36.609265 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.610384 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.612254 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.612928 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.613024 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.618814 (+ 5790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:36.618829 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:36.618845 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:36.618883 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:36.623026 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.623902 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.623911 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.623914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.624125 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.624582 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.624587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.624590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.625018 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:36.625392 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.625693 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.625922 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:24:36.627178 18782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:36.722626 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18644
I20260811 06:24:36.746052 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18794
W20260811 06:24:36.745970 18559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.4.129:33629 (error 108)
I20260811 06:24:36.748080 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.747613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52958)
0811 06:24:36.747798 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.747912 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 06:24:36.748337 18559 catalog_manager.cc:4584] TS 95067b7a7fa141278aa4e0c0e59c0a7f (127.7.4.129:33629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46891fa0e1524ec38941ac592147ec7c: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Aug 11 06:24:36 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:24:36.766214 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.747267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.7.4.130:46879)
0811 06:24:36.747487 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.747493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.747509 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:24:36.747589 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.747593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.747679 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.747779 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:36.749222 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.750395 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.752345 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.753057 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.753173 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.758532 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:36.758547 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:36.758564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:36.758618 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:36.762788 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.763612 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.763621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.763625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.763798 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.764185 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.764190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.764194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.764721 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:36.765068 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.765349 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.765630 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:24:36.766934 18932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:36.801769 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.801228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52966)
0811 06:24:36.801470 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.801578 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 06:24:36.802843 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.802413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52972)
0811 06:24:36.802615 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.802712 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:24:36.863209 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.862692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52984)
0811 06:24:36.862903 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.863010 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:24:36.864324 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.863914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52988)
0811 06:24:36.864079 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.864163 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:24:36.886337 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18794
I20260811 06:24:36.910969 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18943
Aug 11 06:24:36 dist-test-slave-07c2 krb5kdc[18528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:24:36.925591 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.911824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44656 (local address 127.7.4.131:34013)
0811 06:24:36.912023 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:24:36.912028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:24:36.912285 (+ 257us) server_negotiation.cc:407] Connection header received
0811 06:24:36.912429 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:24:36.912433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:24:36.912496 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:24:36.912587 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:24:36.913760 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.914441 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.916031 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:24:36.916499 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:24:36.916591 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:24:36.920661 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:24:36.920673 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:24:36.920687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:24:36.920719 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:24:36.923190 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.923878 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.923885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.923888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.924038 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:24:36.924372 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:24:36.924375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:24:36.924377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:24:36.924709 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:24:36.924923 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:24:36.925190 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:24:36.925341 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:24:36.926270 19076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:36.944823 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.944390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52992)
0811 06:24:36.944603 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.944686 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:24:36.945876 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:36.945465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:52996)
0811 06:24:36.945670 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:36.945754 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 06:24:37.014559 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18943
I20260811 06:24:37.035902 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 18541
I20260811 06:24:37.037161 18611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:24:37.149866 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:24:37.149084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:33629 (local address 127.0.0.1:53004)
0811 06:24:37.149367 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 06:24:37.149516 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:33629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:24:37.150410 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 18541
2026-08-11T06:24:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4402 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:24:37 dist-test-slave-07c2 krb5kdc[19144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:24:37 dist-test-slave-07c2 krb5kdc[19144](info): set up 2 sockets
Aug 11 06:24:37 dist-test-slave-07c2 krb5kdc[19144](info): commencing operation
krb5kdc: starting...
W20260811 06:24:38.328876 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.006s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:24:38 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:24:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:24:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangeradmin_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangeradmin_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangerlookup_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangerlookup_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
I20260811 06:24:38.708439 7186 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5M28Km/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/postgres -l logfile start
2026-08-11 06:24:41.021 UTC [19176] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:24:41.021 UTC [19176] LOG: listening on IPv4 address "127.7.4.148", port 33289
2026-08-11 06:24:41.025 UTC [19176] LOG: listening on Unix socket "/tmp/.s.PGSQL.33289"
2026-08-11 06:24:41.051 UTC [19179] LOG: database system was shut down at 2026-08-11 06:24:39 UTC
2026-08-11 06:24:41.054 UTC [19176] LOG: database system is ready to accept connections
I20260811 06:24:42.095917 7186 mini_postgres.cc:96] Postgres bound to 33289
2026-08-11 06:24:42.122 UTC [19187] FATAL: database "slave" does not exist
127.7.4.148:33289 - accepting connections
I20260811 06:24:42.123060 7186 mini_ranger.cc:161] Starting Ranger...
I20260811 06:24:42.154984 7186 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:24:42.585881 7186 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:24:42.586078 7186 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin
2026-08-11 06:24:42,932 [I] DB FLAVOR :POSTGRES
2026-08-11 06:24:42,933 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:24:42,933 [I] Checking connection..
2026-08-11 06:24:42,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:24:43,202 [I] Checking connection passed.
2026-08-11 06:24:43,202 [I] --------- Verifying version history table ---------
2026-08-11 06:24:43,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:24:43,443 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:24:43,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:24:43,691 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:24:43,692 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:24:43,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:24:43.930 UTC [19259] WARNING: there is no transaction in progress
2026-08-11 06:24:43,937 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:24:43,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:24:44,199 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:24:44,200 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:24:44,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:24:44,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:24:44,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-11 06:24:44,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:24:45,203 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:24:45,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:24:45,466 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:24:45,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:24:45,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:24:45,951 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:24:45,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:24:46.877 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:46.892 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:46.907 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:46.921 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:46.939 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:46.955 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.008 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.014 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.027 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.037 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.046 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.060 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.069 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.075 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.081 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47.087 UTC [19375] WARNING: there is no transaction in progress
2026-08-11 06:24:47,748 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:24:47,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:24:48,001 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:24:48,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:24:48,250 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:24:48,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:24:48,492 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:24:48,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:24:48,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:24:49,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-11 06:24:49,250 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:24:49,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:24:49,505 [I] DB_PATCHES have already been applied
I20260811 06:24:49.511080 7186 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5M28Km/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:24:49.511183 7186 mini_ranger.cc:193] Using host: 127.7.4.148
I20260811 06:24:49.534300 7186 mini_ranger.cc:227] Ranger admin URL: http://127.7.4.148:54263
Aug 11, 2026 6:24:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:24:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:24:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:24:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:24:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:24:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:24:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:24:50.735807 7186 mini_ranger.cc:160] Time spent starting Ranger: real 8.613s user 0.006s sys 0.063s
Aug 11, 2026 6:25:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:25:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 6:25:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:25:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:25:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:25:13.744304 7186 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangerkms_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/rangerkms_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.4.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.4.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:25:13 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:25:13.962163 7186 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:25:13.989874 7186 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:25:14.943912 7186 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:25:14.944064 7186 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms
I20260811 06:25:14.944238 7186 mini_ranger_kms.cc:227] Using postgres at 127.7.4.148:33289
2026-08-11 06:25:15,483 [I] DB FLAVOR :POSTGRES
2026-08-11 06:25:15,483 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:25:15,483 [I] Checking connection
2026-08-11 06:25:15,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:25:15,765 [I] connection success
2026-08-11 06:25:15,765 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:25:15,765 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:25:15,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:25:16,059 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:25:16,059 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:25:16,060 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:25:16,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:33289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:25:16,356 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:25:16.883343 7186 mini_ranger_kms.cc:310] Created kms service
I20260811 06:25:17.092283 7186 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:25:17.148492 7186 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:25:17 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:25:17.571985 7186 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:25:17.574126 7186 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms:/tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:25:17.574259 7186 mini_ranger_kms.cc:240] Using host: 127.7.4.148
I20260811 06:25:17.604061 7186 mini_ranger_kms.cc:276] Ranger KMS PID: 19564
I20260811 06:25:17.604285 7186 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.4.148:54795
Aug 11, 2026 6:25:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:25:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:25:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:25:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:25:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:25:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:25:19 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:25:20.185536 7186 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.223s user 0.114s sys 0.449s
I20260811 06:25:20.185956 7186 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:25:20.186220 7186 mini_ranger_kms.cc:401] 127.7.4.148:54795/kms/v1/keys
Aug 11 06:25:20 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
Aug 11 06:25:20 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429519, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
Aug 11 06:25:23 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:25:23.280072 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46291
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46291
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:54795/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:25:23.641790 19617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:25:23.642061 19617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:25:23.642150 19617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:25:23.649780 19617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:25:23.649874 19617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:25:23.649919 19617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:25:23.649971 19617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:25:23.650063 19617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:25:23.660238 19617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.4.148:54795/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46291
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:25:23.661471 19617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:25:23.662880 19617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:25:23.675618 19622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:23.675565 19623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:23.675653 19625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:25:23.676900 19617 server_base.cc:1034] running on GCE node
I20260811 06:25:23.677754 19617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:25:23.679128 19617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:25:23.680367 19617 hybrid_clock.cc:648] HybridClock initialized: now 1786429523680334 us; error 56 us; skew 500 ppm
I20260811 06:25:23.680795 19617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:25:23 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:25:23.684120 19617 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:25:23.685616 19617 webserver.cc:460] Webserver started at http://127.7.4.190:35805/ using document root <none> and password file <none>
I20260811 06:25:23.686254 19617 fs_manager.cc:359] Metadata directory not provided
I20260811 06:25:23.686393 19617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:25:23.686710 19617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:25:23 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:25:23.740582 19617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "44128b8377b04ac48dd8c8563d35966b"
format_stamp: "Formatted at 2026-08-11 06:25:23 on dist-test-slave-07c2"
server_key: "495ef34e6e543c65797c25e9d2255cf0"
server_key_iv: "563b12bfbfa794dcce3c9b233cc26979"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:23.741660 19617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "44128b8377b04ac48dd8c8563d35966b"
format_stamp: "Formatted at 2026-08-11 06:25:23 on dist-test-slave-07c2"
server_key: "495ef34e6e543c65797c25e9d2255cf0"
server_key_iv: "563b12bfbfa794dcce3c9b233cc26979"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:23.748855 19617 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260811 06:25:23.770115 19632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:23.771909 19617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:25:23.772141 19617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "44128b8377b04ac48dd8c8563d35966b"
format_stamp: "Formatted at 2026-08-11 06:25:23 on dist-test-slave-07c2"
server_key: "495ef34e6e543c65797c25e9d2255cf0"
server_key_iv: "563b12bfbfa794dcce3c9b233cc26979"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:23.772423 19617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:25:23.793476 19617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:25:23.797772 19617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:25:23.798133 19617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:25:23.819733 19617 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:46291
I20260811 06:25:23.819801 19689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:46291 every 8 connection(s)
I20260811 06:25:23.821115 19617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:25:23.825058 19691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:25:23.830515 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 19617
I20260811 06:25:23.830747 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:25:23.831456 7186 external_mini_cluster.cc:1428] Setting key 6374d964447e164f53560fc3f80f76da
I20260811 06:25:23.836349 19691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: Bootstrap starting.
I20260811 06:25:23.840216 19691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: Neither blocks nor log segments found. Creating new log.
I20260811 06:25:23.841454 19691 log.cc:824] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:25:23.844887 19691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: No bootstrap required, opened a new log
Aug 11 06:25:23 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:25:23.854189 19691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } }
I20260811 06:25:23.854743 19691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:25:23.854981 19691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44128b8377b04ac48dd8c8563d35966b, State: Initialized, Role: FOLLOWER
I20260811 06:25:23.855660 19691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } }
I20260811 06:25:23.855924 19691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:25:23.856086 19691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:25:23.856277 19691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:25:23.856596 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:23.833885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.7.4.190:46291)
0811 06:25:23.835006 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:23.835022 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:23.835747 (+ 725us) server_negotiation.cc:407] Connection header received
0811 06:25:23.837829 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:23.837849 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:23.838224 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:23.838519 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:23.840193 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:23.841664 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:23.843236 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:23.844007 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:23.844149 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:23.848753 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:23.848761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:23.848773 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:23.848810 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:23.852074 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:23.852757 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:23.852763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:23.852772 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:23.852916 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:23.853373 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:23.853377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:23.853379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:23.853871 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:23.854271 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:23.854875 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 06:25:23.855422 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":608,"threads_started":1}
I20260811 06:25:23.858929 19691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } }
I20260811 06:25:23.859516 19691 leader_election.cc:302] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44128b8377b04ac48dd8c8563d35966b; no voters:
I20260811 06:25:23.860114 19691 leader_election.cc:288] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:25:23.860246 19697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:25:23.861063 19697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [term 1 LEADER]: Becoming Leader. State: Replica: 44128b8377b04ac48dd8c8563d35966b, State: Running, Role: LEADER
I20260811 06:25:23.861660 19697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } }
I20260811 06:25:23.862218 19691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:25:23.865249 19699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44128b8377b04ac48dd8c8563d35966b. Latest consensus state: current_term: 1 leader_uuid: "44128b8377b04ac48dd8c8563d35966b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } } }
I20260811 06:25:23.865686 19699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [sys.catalog]: This master's current role is: LEADER
I20260811 06:25:23.866182 19698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44128b8377b04ac48dd8c8563d35966b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44128b8377b04ac48dd8c8563d35966b" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46291 } } }
I20260811 06:25:23.866617 19698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b [sys.catalog]: This master's current role is: LEADER
I20260811 06:25:23.872781 19704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:25:23.878458 19704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:25:23.879174 19691 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:25:23.898185 19704 catalog_manager.cc:1347] Generated new cluster ID: 17dd809fe5934ae0a62619cbe80d3704
I20260811 06:25:23.898406 19704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:25:23.919359 19704 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:25:23.920738 19704 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:25:23.929214 19704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: Generated new TSK 0
I20260811 06:25:23.930131 19704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:25:24.693212 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:54795/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:25:24 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429524, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:25:25.094720 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:25:25.095018 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:25:25.095206 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:25:25.102680 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:25:25.102814 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:25:25.102985 19741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:25:25.114840 19741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.4.148:54795/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:25:25.116269 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:25:25.117712 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:25:25.136919 19752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:25.138186 19753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:25.146569 19757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:25:25.146835 19741 server_base.cc:1034] running on GCE node
I20260811 06:25:25.147750 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:25:25.151600 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:25:25.152874 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786429525152804 us; error 69 us; skew 500 ppm
I20260811 06:25:25.153204 19741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:25:25 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:25:25.156857 19741 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:25:25.159667 19741 webserver.cc:460] Webserver started at http://127.7.4.129:41853/ using document root <none> and password file <none>
I20260811 06:25:25.160375 19741 fs_manager.cc:359] Metadata directory not provided
I20260811 06:25:25.160521 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:25:25.160907 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:25:25 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:25:25.185675 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "1fc382dc5c2b430db7264c1d7aecf190"
format_stamp: "Formatted at 2026-08-11 06:25:25 on dist-test-slave-07c2"
server_key: "e5c80fdd2ff4a817aaac2749b9f89396"
server_key_iv: "c5f7ad25c376abd1780df1c978db4e3e"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:25.188290 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "1fc382dc5c2b430db7264c1d7aecf190"
format_stamp: "Formatted at 2026-08-11 06:25:25 on dist-test-slave-07c2"
server_key: "e5c80fdd2ff4a817aaac2749b9f89396"
server_key_iv: "c5f7ad25c376abd1780df1c978db4e3e"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:25.194615 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 06:25:25.224566 19764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:25.226244 19741 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.004s
I20260811 06:25:25.226488 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "1fc382dc5c2b430db7264c1d7aecf190"
format_stamp: "Formatted at 2026-08-11 06:25:25 on dist-test-slave-07c2"
server_key: "e5c80fdd2ff4a817aaac2749b9f89396"
server_key_iv: "c5f7ad25c376abd1780df1c978db4e3e"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:25.226737 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:25:25.244633 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:25:25.250339 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:25:25.250749 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:25:25.252206 19741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:25:25.254168 19741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:25:25.254262 19741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:25.254395 19741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:25:25.254468 19741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:25.342350 19741 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39211
I20260811 06:25:25.342370 19888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39211 every 8 connection(s)
I20260811 06:25:25.343888 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:25:25.348695 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 19741
I20260811 06:25:25.348906 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:25:25.349974 7186 external_mini_cluster.cc:1428] Setting key cfe225f705de823d80860d6393d2b9bc
Aug 11 06:25:25 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429525, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:25:25.384601 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:25.348750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:50493 (local address 127.7.4.190:46291)
0811 06:25:25.353226 (+ 4476us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:25.353232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:25.359201 (+ 5969us) server_negotiation.cc:407] Connection header received
0811 06:25:25.359271 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:25.359278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:25.359390 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:25.359497 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:25.362631 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:25.363882 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:25.366554 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:25.367335 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:25.367450 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:25.376422 (+ 8972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:25.376431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:25.376438 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:25.376485 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:25.380487 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:25.381498 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:25.381506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:25.381513 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:25.381696 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:25.382230 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:25.382237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:25.382243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:25.382640 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:25.382769 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:25.383859 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0811 06:25:25.384136 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4355,"thread_start_us":4194,"threads_started":1}
I20260811 06:25:25.387480 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:25.349317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46291 (local address 127.7.4.129:50493)
0811 06:25:25.355272 (+ 5955us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:25.355932 (+ 660us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:25.358835 (+ 2903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:25.360712 (+ 1877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:25.360726 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:25.361558 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:25:25.362419 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:25.362448 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:25.364132 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:25.364140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:25.366370 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:25.366387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:25.367804 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:25.367812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:25.368144 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:25.369036 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:25:25.369080 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:25:25.376140 (+ 7060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:25:25.380668 (+ 4528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:25.380679 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:25.380699 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:25.381338 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:25.381849 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:25.381856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:25.381862 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:25.382103 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:25.382897 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:25:25.382907 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:25:25.383375 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 06:25:25.385195 (+ 1820us) client_negotiation.cc:240] Negotiation successful
0811 06:25:25.385822 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5644,"thread_start_us":6823,"threads_started":1}
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
I20260811 06:25:25.389712 19889 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46291
I20260811 06:25:25.390038 19889 heartbeater.cc:464] Registering TS with master...
I20260811 06:25:25.390882 19889 heartbeater.cc:511] Master 127.7.4.190:46291 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
I20260811 06:25:25.393743 19649 ts_manager.cc:194] Registered new tserver with Master: 1fc382dc5c2b430db7264c1d7aecf190 (127.7.4.129:39211)
I20260811 06:25:25.399715 19649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:50493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:25:25.432010 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:54795/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:25:25 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429524, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
Aug 11 06:25:25 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429524, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
W20260811 06:25:25.943045 19895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:25:25.943575 19895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:25:25.943787 19895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:25:25.955168 19895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:25:25.955399 19895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:25:25.955665 19895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:25:25.971553 19895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.4.148:54795/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:25:25.973467 19895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:25:25.975186 19895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:25:26.009193 19904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:25:26.013312 19895 server_base.cc:1034] running on GCE node
W20260811 06:25:26.013947 19903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:26.015573 19908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:25:26.016450 19895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:25:26.026577 19895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:25:26.027994 19895 hybrid_clock.cc:648] HybridClock initialized: now 1786429526027889 us; error 112 us; skew 500 ppm
I20260811 06:25:26.028447 19895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:25:26.039661 19895 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:25:26.083837 19895 webserver.cc:460] Webserver started at http://127.7.4.130:46027/ using document root <none> and password file <none>
I20260811 06:25:26.085084 19895 fs_manager.cc:359] Metadata directory not provided
I20260811 06:25:26.085361 19895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:25:26.086018 19895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:25:26.117976 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "5704142df1a55d169dbe2906872484af"
server_key_iv: "2a7689c5262194b9168b59d1d67bb2b1"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.119089 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "5704142df1a55d169dbe2906872484af"
server_key_iv: "2a7689c5262194b9168b59d1d67bb2b1"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.126739 19895 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 06:25:26.146529 19922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.148329 19895 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:25:26.148517 19895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "5704142df1a55d169dbe2906872484af"
server_key_iv: "2a7689c5262194b9168b59d1d67bb2b1"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.148955 19895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:25:26.182174 19895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:25:26.186493 19895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:25:26.186823 19895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:25:26.187866 19895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:25:26.189687 19895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:25:26.189772 19895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.189903 19895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:25:26.189955 19895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.228406 19895 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:39081
I20260811 06:25:26.228442 20044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:39081 every 8 connection(s)
I20260811 06:25:26.229789 19895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:25:26.239689 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 19895
I20260811 06:25:26.239881 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:25:26.240353 7186 external_mini_cluster.cc:1428] Setting key 7d2e3e07db8f773cb794032cad0eae85
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:25:26.250325 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.232773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:55761 (local address 127.7.4.190:46291)
0811 06:25:26.233249 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.233253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.234180 (+ 927us) server_negotiation.cc:407] Connection header received
0811 06:25:26.236272 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.236280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.236385 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.236508 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:26.237911 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.238650 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.240379 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.240810 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.240950 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.244480 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:26.244485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:26.244488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:26.244518 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:26.247361 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.248033 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.248038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.248043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.248166 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.248613 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.248618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.248621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.248906 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:26.249064 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.249731 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.249962 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":260,"threads_started":1}
I20260811 06:25:26.251190 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.233178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46291 (local address 127.7.4.130:55761)
0811 06:25:26.233994 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:26.234035 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:26.235979 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:26.236733 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:26.236741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:26.237187 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:25:26.237735 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.237750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.238821 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.238825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.240234 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.240246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.241038 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.241046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.241279 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.241963 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:25:26.241986 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:25:26.244271 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:25:26.247487 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:26.247496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:26.247510 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:26.247904 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:26.248276 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:26.248281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:26.248285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:26.248475 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:26.249037 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:25:26.249044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:25:26.249358 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 06:25:26.249610 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 06:25:26.250087 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":376,"threads_started":1}
I20260811 06:25:26.252573 20045 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46291
I20260811 06:25:26.252875 20045 heartbeater.cc:464] Registering TS with master...
I20260811 06:25:26.253679 20045 heartbeater.cc:511] Master 127.7.4.190:46291 requested a full tablet report, sending...
I20260811 06:25:26.255172 19649 ts_manager.cc:194] Registered new tserver with Master: e4fa0d3dc4674eb9917f1c6b42a18b1a (127.7.4.130:39081)
I20260811 06:25:26.256093 19649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:55761
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:25:26.304292 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:54795/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:25:26.403787 19889 heartbeater.cc:503] Master 127.7.4.190:46291 was elected leader, sending a full tablet report...
W20260811 06:25:26.668515 20051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:25:26.668804 20051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:25:26.668896 20051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:25:26.677196 20051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:25:26.677385 20051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:25:26.677579 20051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:25:26.689363 20051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.4.148:54795/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:25:26.690691 20051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:25:26.692104 20051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:25:26.707721 20057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:26.707793 20058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:25:26.708005 20061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:25:26.709996 20051 server_base.cc:1034] running on GCE node
I20260811 06:25:26.710775 20051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:25:26.712021 20051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:25:26.713349 20051 hybrid_clock.cc:648] HybridClock initialized: now 1786429526713216 us; error 97 us; skew 500 ppm
I20260811 06:25:26.713784 20051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:25:26.718019 20051 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:25:26.719863 20051 webserver.cc:460] Webserver started at http://127.7.4.131:35103/ using document root <none> and password file <none>
I20260811 06:25:26.720605 20051 fs_manager.cc:359] Metadata directory not provided
I20260811 06:25:26.720768 20051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:25:26.721091 20051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:25:26.739202 20051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "72e1576d9b8b4b82947f5c5df090ae8b"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "c9137b8cf7b64c69b850d39e3a60fd41"
server_key_iv: "2d7415068d0b944ee4f398429a0fd1b5"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.740552 20051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "72e1576d9b8b4b82947f5c5df090ae8b"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "c9137b8cf7b64c69b850d39e3a60fd41"
server_key_iv: "2d7415068d0b944ee4f398429a0fd1b5"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.747839 20051 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.001s
I20260811 06:25:26.771826 20069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.773597 20051 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.001s
I20260811 06:25:26.773784 20051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "72e1576d9b8b4b82947f5c5df090ae8b"
format_stamp: "Formatted at 2026-08-11 06:25:26 on dist-test-slave-07c2"
server_key: "c9137b8cf7b64c69b850d39e3a60fd41"
server_key_iv: "2d7415068d0b944ee4f398429a0fd1b5"
server_key_version: "kuduclusterkey@0"
I20260811 06:25:26.774015 20051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:25:26.785634 20051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:25:26.789880 20051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:25:26.790194 20051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:25:26.791386 20051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:25:26.792994 20051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:25:26.793099 20051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.793264 20051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:25:26.793339 20051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:25:26.840104 20051 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:41781
I20260811 06:25:26.840201 20203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:41781 every 8 connection(s)
I20260811 06:25:26.841428 20051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:25:26.851750 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 20051
I20260811 06:25:26.851943 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:25:26.852427 7186 external_mini_cluster.cc:1428] Setting key e33951a6dd9c6643927af9b4104ad76b
Aug 11 06:25:26 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429526, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:25:26.861522 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.844263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:44329 (local address 127.7.4.190:46291)
0811 06:25:26.844843 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.844848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.845759 (+ 911us) server_negotiation.cc:407] Connection header received
0811 06:25:26.847620 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.847626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.847741 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.847865 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:26.849262 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.850022 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.851665 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.852190 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.852331 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.856040 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:26.856044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:26.856048 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:26.856077 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:26.858484 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.859190 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.859197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.859205 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.859385 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.859837 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.859842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.859845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.860124 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:26.860290 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.860913 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.861124 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":355,"threads_started":1}
I20260811 06:25:26.862317 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.844833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46291 (local address 127.7.4.131:44329)
0811 06:25:26.845565 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:26.845622 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:26.847352 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:26.848092 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:26.848100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:26.848526 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:25:26.849083 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.849099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.850182 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.850189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.851504 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.851516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.852415 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.852422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.852704 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.853464 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:25:26.853486 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:25:26.855824 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:25:26.858633 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:26.858641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:26.858664 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:26.859047 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:26.859505 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:25:26.859509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:25:26.859519 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:25:26.859712 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:25:26.860263 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:25:26.860270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:25:26.860567 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 06:25:26.860811 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 06:25:26.861249 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":330,"threads_started":1}
I20260811 06:25:26.863620 20204 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46291
I20260811 06:25:26.863878 20204 heartbeater.cc:464] Registering TS with master...
I20260811 06:25:26.864472 20204 heartbeater.cc:511] Master 127.7.4.190:46291 requested a full tablet report, sending...
I20260811 06:25:26.865826 19649 ts_manager.cc:194] Registered new tserver with Master: 72e1576d9b8b4b82947f5c5df090ae8b (127.7.4.131:41781)
I20260811 06:25:26.866973 19649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:44329
I20260811 06:25:26.868074 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:25:26.885281 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.872695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.7.4.190:46291)
0811 06:25:26.872871 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.872876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.873413 (+ 537us) server_negotiation.cc:407] Connection header received
0811 06:25:26.873558 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.873561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.873630 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.873716 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:26.875152 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.875919 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.877269 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.877885 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.878037 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.880453 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:26.880457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:26.880460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:26.880487 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:26.882943 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.883605 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.883609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.883611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.883733 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:26.884070 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:26.884073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:26.884075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:26.884368 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:26.884542 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.884912 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.885069 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:25:26.889411 19648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:25:26.896790 19648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:25:26.920228 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.909143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:41781 (local address 127.0.0.1:57292)
0811 06:25:26.910180 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:26.910244 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:26.910444 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:26.911995 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:26.912002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:26.912055 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:25:26.912786 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.912796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.915210 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.915216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.917392 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.917404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.919126 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.919134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.919350 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.919408 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 06:25:26.919519 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:25:26.919744 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":54,"spinlock_wait_cycles":1664,"thread_start_us":591,"threads_started":1}
I20260811 06:25:26.920679 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.910541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:39081 (local address 127.0.0.1:36926)
0811 06:25:26.911146 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:26.911164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:26.911268 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:26.911995 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:26.912002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:26.912055 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:25:26.912758 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.912778 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.915219 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.915223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.917823 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.917837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.919691 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.919697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.919885 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.919919 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:25:26.919985 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:25:26.920111 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":57,"spinlock_wait_cycles":5760,"thread_start_us":335,"threads_started":1}
I20260811 06:25:26.921161 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.911228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39211 (local address 127.0.0.1:34734)
0811 06:25:26.911793 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:26.911809 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:26.911910 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:26.913966 (+ 2056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:26.913971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:26.913996 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:25:26.914388 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.914396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.916338 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.916348 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.918292 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:26.918302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.920382 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.920395 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:26.920562 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.920595 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:25:26.920693 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:25:26.920842 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":790,"threads_started":1}
I20260811 06:25:26.921590 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.910795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.7.4.130:39081)
0811 06:25:26.911384 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.911391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.911416 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:25:26.911512 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.911518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.911686 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.913094 (+ 1408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:25:26.913408 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.915017 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.917978 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.919457 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.920768 (+ 1311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.920962 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.921031 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.921174 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":341,"threads_started":1}
I20260811 06:25:26.921720 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.911852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.7.4.129:39211)
0811 06:25:26.913527 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.913533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.913553 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:25:26.913624 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.913628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.913780 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.913985 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:25:26.914547 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.916208 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.918452 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.919904 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.920070 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.920598 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.921014 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.921185 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1535,"thread_start_us":1335,"threads_started":1}
I20260811 06:25:26.921984 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:26.909435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.7.4.131:41781)
0811 06:25:26.910200 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:26.910209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:26.910349 (+ 140us) server_negotiation.cc:407] Connection header received
0811 06:25:26.911419 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:26.911428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:26.911739 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:26.912932 (+ 1193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:25:26.913251 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.915049 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.917544 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:26.918843 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:26.920716 (+ 1873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:26.921148 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:26.921395 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 06:25:26.921519 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":428,"threads_started":1}
I20260811 06:25:26.925386 19974 tablet_service.cc:1467] Processing CreateTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 (DEFAULT_TABLE table=test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:25:26.925431 19817 tablet_service.cc:1467] Processing CreateTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 (DEFAULT_TABLE table=test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:25:26.925858 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 (DEFAULT_TABLE table=test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:25:26.927173 19974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d64435b2504614a80cbb6fbe2c9155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:25:26.927201 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d64435b2504614a80cbb6fbe2c9155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:25:26.929641 19817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15d64435b2504614a80cbb6fbe2c9155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:25:26.941620 20225 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Bootstrap starting.
I20260811 06:25:26.944329 20226 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Bootstrap starting.
I20260811 06:25:26.946015 20225 tablet_bootstrap.cc:654] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Neither blocks nor log segments found. Creating new log.
I20260811 06:25:26.946846 20227 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Bootstrap starting.
I20260811 06:25:26.947535 20225 log.cc:824] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:25:26.948444 20226 tablet_bootstrap.cc:654] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Neither blocks nor log segments found. Creating new log.
I20260811 06:25:26.949549 20226 log.cc:824] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:25:26.950373 20227 tablet_bootstrap.cc:654] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Neither blocks nor log segments found. Creating new log.
I20260811 06:25:26.950970 20225 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: No bootstrap required, opened a new log
I20260811 06:25:26.951273 20225 ts_tablet_manager.cc:1397] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 06:25:26.951658 20227 log.cc:824] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Log is configured to *not* fsync() on all Append() calls
I20260811 06:25:26.952637 20226 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: No bootstrap required, opened a new log
I20260811 06:25:26.952899 20226 ts_tablet_manager.cc:1397] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:25:26.961319 20227 tablet_bootstrap.cc:492] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: No bootstrap required, opened a new log
I20260811 06:25:26.961580 20227 ts_tablet_manager.cc:1397] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260811 06:25:26.961973 20225 raft_consensus.cc:348] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.962488 20225 raft_consensus.cc:374] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:25:26.962591 20225 raft_consensus.cc:729] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fa0d3dc4674eb9917f1c6b42a18b1a, State: Initialized, Role: FOLLOWER
I20260811 06:25:26.963377 20225 consensus_queue.cc:260] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.963413 20226 raft_consensus.cc:348] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.963811 20226 raft_consensus.cc:374] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:25:26.963936 20226 raft_consensus.cc:729] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e1576d9b8b4b82947f5c5df090ae8b, State: Initialized, Role: FOLLOWER
I20260811 06:25:26.964620 20226 consensus_queue.cc:260] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.965909 20204 heartbeater.cc:503] Master 127.7.4.190:46291 was elected leader, sending a full tablet report...
I20260811 06:25:26.966790 20226 ts_tablet_manager.cc:1428] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 06:25:26.970656 20045 heartbeater.cc:503] Master 127.7.4.190:46291 was elected leader, sending a full tablet report...
I20260811 06:25:26.971671 20225 ts_tablet_manager.cc:1428] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260811 06:25:26.971896 20227 raft_consensus.cc:348] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.972312 20227 raft_consensus.cc:374] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:25:26.972424 20227 raft_consensus.cc:729] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc382dc5c2b430db7264c1d7aecf190, State: Initialized, Role: FOLLOWER
I20260811 06:25:26.972999 20227 consensus_queue.cc:260] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:26.974627 20227 ts_tablet_manager.cc:1428] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
W20260811 06:25:26.982286 20046 tablet.cc:2369] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:25:27.093619 20206 tablet.cc:2369] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:25:27.112017 19890 tablet.cc:2369] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:25:27.151773 20234 raft_consensus.cc:482] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:25:27.152104 20234 raft_consensus.cc:504] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:27.153803 20234 leader_election.cc:288] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72e1576d9b8b4b82947f5c5df090ae8b (127.7.4.131:41781), e4fa0d3dc4674eb9917f1c6b42a18b1a (127.7.4.130:39081)
I20260811 06:25:27.160825 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.154259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:47471 (local address 127.7.4.131:41781)
0811 06:25:27.154493 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.154498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.154916 (+ 418us) server_negotiation.cc:407] Connection header received
0811 06:25:27.155226 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.155232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.155372 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.155432 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:25:27.156145 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.156862 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.158971 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.159831 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.159985 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.160101 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.160422 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.160546 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:25:27.160853 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.155126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:33937 (local address 127.7.4.130:39081)
0811 06:25:27.155372 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.155377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.155630 (+ 253us) server_negotiation.cc:407] Connection header received
0811 06:25:27.155855 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.155860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.155930 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.156035 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:25:27.156590 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.157277 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.159012 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.159851 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.160000 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.160103 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.160452 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.160573 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 06:25:27.161015 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.154868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:39081 (local address 127.7.4.129:33937)
0811 06:25:27.155506 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:27.155536 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:27.155710 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:27.156052 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:27.156056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:27.156080 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:25:27.156438 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:27.156446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.157422 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.157428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:27.158852 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:27.158864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.160101 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.160108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:27.160293 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.160317 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:25:27.160424 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:25:27.160547 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":43,"thread_start_us":297,"threads_started":1}
I20260811 06:25:27.161392 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.154049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:41781 (local address 127.7.4.129:47471)
0811 06:25:27.154785 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 06:25:27.154811 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:25:27.154969 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:25:27.155678 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:25:27.155686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:25:27.155721 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:25:27.156000 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:27.156007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.157003 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.157008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:27.158811 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:25:27.158823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.160063 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.160069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:25:27.160252 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.160284 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:25:27.160394 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:25:27.160519 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":24,"thread_start_us":361,"threads_started":1}
I20260811 06:25:27.162209 19998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d64435b2504614a80cbb6fbe2c9155" candidate_uuid: "1fc382dc5c2b430db7264c1d7aecf190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" is_pre_election: true
I20260811 06:25:27.162427 20147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d64435b2504614a80cbb6fbe2c9155" candidate_uuid: "1fc382dc5c2b430db7264c1d7aecf190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" is_pre_election: true
I20260811 06:25:27.162783 19998 raft_consensus.cc:2454] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc382dc5c2b430db7264c1d7aecf190 in term 0.
I20260811 06:25:27.162912 20147 raft_consensus.cc:2454] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc382dc5c2b430db7264c1d7aecf190 in term 0.
I20260811 06:25:27.163522 19766 leader_election.cc:302] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc382dc5c2b430db7264c1d7aecf190, e4fa0d3dc4674eb9917f1c6b42a18b1a; no voters:
I20260811 06:25:27.163949 20234 raft_consensus.cc:2781] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:25:27.164121 20234 raft_consensus.cc:482] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:25:27.164252 20234 raft_consensus.cc:3037] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:25:27.166515 20234 raft_consensus.cc:504] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:27.167081 20234 leader_election.cc:288] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [CANDIDATE]: Term 1 election: Requested vote from peers 72e1576d9b8b4b82947f5c5df090ae8b (127.7.4.131:41781), e4fa0d3dc4674eb9917f1c6b42a18b1a (127.7.4.130:39081)
I20260811 06:25:27.167536 20147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d64435b2504614a80cbb6fbe2c9155" candidate_uuid: "1fc382dc5c2b430db7264c1d7aecf190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e1576d9b8b4b82947f5c5df090ae8b"
I20260811 06:25:27.167623 19998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15d64435b2504614a80cbb6fbe2c9155" candidate_uuid: "1fc382dc5c2b430db7264c1d7aecf190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a"
I20260811 06:25:27.167711 20147 raft_consensus.cc:3037] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:25:27.167870 19998 raft_consensus.cc:3037] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:25:27.170091 20147 raft_consensus.cc:2454] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc382dc5c2b430db7264c1d7aecf190 in term 1.
I20260811 06:25:27.170480 19766 leader_election.cc:302] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc382dc5c2b430db7264c1d7aecf190, 72e1576d9b8b4b82947f5c5df090ae8b; no voters:
I20260811 06:25:27.170509 19998 raft_consensus.cc:2454] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc382dc5c2b430db7264c1d7aecf190 in term 1.
I20260811 06:25:27.170754 20234 raft_consensus.cc:2781] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:25:27.171270 20234 raft_consensus.cc:686] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 1 LEADER]: Becoming Leader. State: Replica: 1fc382dc5c2b430db7264c1d7aecf190, State: Running, Role: LEADER
I20260811 06:25:27.171792 20234 consensus_queue.cc:237] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } }
I20260811 06:25:27.176254 19648 catalog_manager.cc:5495] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc382dc5c2b430db7264c1d7aecf190 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "1fc382dc5c2b430db7264c1d7aecf190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc382dc5c2b430db7264c1d7aecf190" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39211 } health_report { overall_health: HEALTHY } } }
I20260811 06:25:27.218214 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.212925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.7.4.129:39211)
0811 06:25:27.213176 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.213181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.213199 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:25:27.213292 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.213296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.213375 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.213511 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:25:27.213845 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.214715 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.216274 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.216669 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.216761 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.217237 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:25:27.217738 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:25:27.217853 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.217936 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.218001 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:25:27.226402 19998 raft_consensus.cc:1261] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 1 FOLLOWER]: Refusing update from remote peer 1fc382dc5c2b430db7264c1d7aecf190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:25:27.226438 20147 raft_consensus.cc:1261] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 1 FOLLOWER]: Refusing update from remote peer 1fc382dc5c2b430db7264c1d7aecf190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:25:27.227393 20238 consensus_queue.cc:1035] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4fa0d3dc4674eb9917f1c6b42a18b1a" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:25:27.230492 20234 consensus_queue.cc:1035] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72e1576d9b8b4b82947f5c5df090ae8b" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 41781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:25:27.247641 20241 mvcc.cc:205] Tried to move back new op lower bound from 7317215343509610496 to 7317215343299665920. Current Snapshot: MvccSnapshot[applied={T|T < 7317215343509610496}]
I20260811 06:25:27.261564 19648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:25:27.261839 19648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:25:27.265489 19648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44128b8377b04ac48dd8c8563d35966b: Sending DeleteTablet for 3 replicas of tablet 15d64435b2504614a80cbb6fbe2c9155
I20260811 06:25:27.266535 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:25:27 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:57292
I20260811 06:25:27.266768 19974 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:25:27 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:36926
I20260811 06:25:27.267058 19817 tablet_service.cc:1514] Processing DeleteTablet for tablet 15d64435b2504614a80cbb6fbe2c9155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:25:27 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:34734
I20260811 06:25:27.269888 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 19741
I20260811 06:25:27.279234 20251 tablet_replica.cc:330] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: stopping tablet replica
I20260811 06:25:27.279510 20252 tablet_replica.cc:330] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: stopping tablet replica
I20260811 06:25:27.279727 20251 raft_consensus.cc:2229] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:25:27.279914 20252 raft_consensus.cc:2229] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:25:27.280301 20251 raft_consensus.cc:2258] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:25:27.280354 20252 raft_consensus.cc:2258] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:25:27.282757 20251 ts_tablet_manager.cc:1905] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:25:27.282817 20252 ts_tablet_manager.cc:1905] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:25:27.292475 20252 ts_tablet_manager.cc:1918] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:25:27.292708 20252 log.cc:1196] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/15d64435b2504614a80cbb6fbe2c9155
I20260811 06:25:27.293376 20252 ts_tablet_manager.cc:1939] T 15d64435b2504614a80cbb6fbe2c9155 P e4fa0d3dc4674eb9917f1c6b42a18b1a: Deleting consensus metadata
I20260811 06:25:27.295966 19634 catalog_manager.cc:4841] TS e4fa0d3dc4674eb9917f1c6b42a18b1a (127.7.4.130:39081): tablet 15d64435b2504614a80cbb6fbe2c9155 (table test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]) successfully deleted
I20260811 06:25:27.298588 20253 tablet_replica.cc:330] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: stopping tablet replica
I20260811 06:25:27.299139 20253 raft_consensus.cc:2229] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:25:27.300067 20253 raft_consensus.cc:2258] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:25:27.300966 20251 ts_tablet_manager.cc:1918] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:25:27.301183 20251 log.cc:1196] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/15d64435b2504614a80cbb6fbe2c9155
I20260811 06:25:27.301687 20251 ts_tablet_manager.cc:1939] T 15d64435b2504614a80cbb6fbe2c9155 P 72e1576d9b8b4b82947f5c5df090ae8b: Deleting consensus metadata
I20260811 06:25:27.302495 20253 ts_tablet_manager.cc:1905] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:25:27.304001 19634 catalog_manager.cc:4841] TS 72e1576d9b8b4b82947f5c5df090ae8b (127.7.4.131:41781): tablet 15d64435b2504614a80cbb6fbe2c9155 (table test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]) successfully deleted
Aug 11 06:25:27 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:25:27.310374 20253 ts_tablet_manager.cc:1918] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:25:27.310575 20253 log.cc:1196] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/15d64435b2504614a80cbb6fbe2c9155
I20260811 06:25:27.311422 20253 ts_tablet_manager.cc:1939] T 15d64435b2504614a80cbb6fbe2c9155 P 1fc382dc5c2b430db7264c1d7aecf190: Deleting consensus metadata
I20260811 06:25:27.313741 19634 catalog_manager.cc:4841] TS 1fc382dc5c2b430db7264c1d7aecf190 (127.7.4.129:39211): tablet 15d64435b2504614a80cbb6fbe2c9155 (table test-table [id=dcd823b54b5e4fa6816f5cff0b0eb387]) successfully deleted
I20260811 06:25:27.316892 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.271242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.7.4.129:39211)
0811 06:25:27.271460 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.271466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.294118 (+ 22652us) server_negotiation.cc:407] Connection header received
0811 06:25:27.294706 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.294712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.294813 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.294900 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:27.298068 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.299106 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.301431 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.302112 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.302225 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.308830 (+ 6605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:27.308847 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:27.308866 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:27.308921 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:27.313603 (+ 4682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.314468 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.314475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.314478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.314630 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.315071 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.315075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.315077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.315552 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:27.315915 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.316228 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.316486 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 06:25:27.317623 19884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:25:27.420176 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 19741
I20260811 06:25:27.443413 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 19895
Aug 11 06:25:27 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:25:27.458393 20221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.444281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.7.4.130:39081)
0811 06:25:27.444482 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.444488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.444508 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:25:27.444676 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.444687 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.444769 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.444890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:27.446150 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.446906 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.448294 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.448703 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.448845 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.453059 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:27.453070 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:27.453084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:27.453114 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:27.456041 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.456639 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.456645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.456647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.456767 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.457131 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.457134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.457136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.457488 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:27.457701 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.457974 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.458126 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:25:27.459106 20039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:25:27.550534 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 19895
I20260811 06:25:27.572325 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 20051
Aug 11 06:25:27 dist-test-slave-07c2 krb5kdc[19144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:25:27.596390 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:25:27.573731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.7.4.131:41781)
0811 06:25:27.574250 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 06:25:27.574258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:25:27.574281 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:25:27.574348 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:25:27.574353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:25:27.574441 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:25:27.574582 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:25:27.576877 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.579198 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.581513 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:25:27.582119 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:25:27.582264 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:25:27.588916 (+ 6652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:25:27.588955 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:25:27.588975 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:25:27.589027 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:25:27.592966 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.593945 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.593956 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.593962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.594125 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:25:27.594527 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:25:27.594533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:25:27.594537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:25:27.595023 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:25:27.595367 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 06:25:27.595721 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 06:25:27.595967 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260811 06:25:27.597263 20199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:25:27.686498 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 20051
I20260811 06:25:27.707715 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 19617
I20260811 06:25:27.708959 19684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:25:27.822916 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 19617
I20260811 06:25:27.852385 7186 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:25:28.197463 7186 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:25:28.197746 7186 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:25:28.552608 7186 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:25:28.552 UTC [19176] LOG: received smart shutdown request
2026-08-11 06:25:28.557 UTC [19176] LOG: background worker "logical replication launcher" (PID 19185) exited with exit code 1
2026-08-11 06:25:28.557 UTC [19180] LOG: shutting down
2026-08-11 06:25:28.704 UTC [19176] LOG: database system is shut down
2026-08-11T06:25:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51683 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:25:28 dist-test-slave-07c2 krb5kdc[20268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:25:28 dist-test-slave-07c2 krb5kdc[20268](info): set up 2 sockets
Aug 11 06:25:28 dist-test-slave-07c2 krb5kdc[20268](info): commencing operation
krb5kdc: starting...
W20260811 06:25:30.009232 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:25:30 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:25:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangeradmin_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangeradmin_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangerlookup_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangerlookup_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
I20260811 06:25:30.367627 7186 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5M28Km/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/postgres -l logfile start
2026-08-11 06:25:32.694 UTC [20303] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:25:32.694 UTC [20303] LOG: listening on IPv4 address "127.7.4.148", port 40845
2026-08-11 06:25:32.697 UTC [20303] LOG: listening on Unix socket "/tmp/.s.PGSQL.40845"
2026-08-11 06:25:32.720 UTC [20307] LOG: database system was shut down at 2026-08-11 06:25:31 UTC
2026-08-11 06:25:32.723 UTC [20303] LOG: database system is ready to accept connections
I20260811 06:25:33.760542 7186 mini_postgres.cc:96] Postgres bound to 40845
2026-08-11 06:25:33.785 UTC [20315] FATAL: database "slave" does not exist
127.7.4.148:40845 - accepting connections
I20260811 06:25:33.786628 7186 mini_ranger.cc:161] Starting Ranger...
I20260811 06:25:33.816982 7186 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:25:34.245937 7186 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:25:34.246032 7186 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin
2026-08-11 06:25:34,543 [I] DB FLAVOR :POSTGRES
2026-08-11 06:25:34,544 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:25:34,544 [I] Checking connection..
2026-08-11 06:25:34,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:25:34,794 [I] Checking connection passed.
2026-08-11 06:25:34,794 [I] --------- Verifying version history table ---------
2026-08-11 06:25:34,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:25:35,031 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:25:35,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:25:35,264 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:25:35,264 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:25:35,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:25:35.503 UTC [20383] WARNING: there is no transaction in progress
2026-08-11 06:25:35,510 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:25:35,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:25:35,764 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:25:35,764 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:25:35,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:25:35,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:25:36,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-11 06:25:36,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:25:36,732 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:25:36,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:25:36,979 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:25:36,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:25:37,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:25:37,488 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:25:37,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:25:38.414 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.429 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.444 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.458 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.472 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.491 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.543 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.549 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.562 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.572 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.581 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.593 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.602 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.608 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.614 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:38.620 UTC [20503] WARNING: there is no transaction in progress
2026-08-11 06:25:39,205 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:25:39,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:25:39,442 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:25:39,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:25:39,701 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:25:39,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:25:39,951 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:25:39,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:25:40,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:25:40,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-11 06:25:40,693 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:25:40,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:25:40,943 [I] DB_PATCHES have already been applied
I20260811 06:25:40.949735 7186 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5M28Km/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:25:40.949831 7186 mini_ranger.cc:193] Using host: 127.7.4.148
I20260811 06:25:40.971539 7186 mini_ranger.cc:227] Ranger admin URL: http://127.7.4.148:50933
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:25:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:25:42.164772 7186 mini_ranger.cc:160] Time spent starting Ranger: real 8.378s user 0.002s sys 0.058s
Aug 11, 2026 6:26:01 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:26:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 6:26:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:26:01 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:26:02 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:26:02.995028 7186 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.4.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.4.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangerkms_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.4.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/rangerkms_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.4.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.4.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.4.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/HTTP_127.7.4.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:26:03 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429563, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:26:03.213692 7186 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:26:03.242784 7186 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:26:04.075881 7186 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:26:04.076011 7186 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms
I20260811 06:26:04.076059 7186 mini_ranger_kms.cc:227] Using postgres at 127.7.4.148:40845
2026-08-11 06:26:04,446 [I] DB FLAVOR :POSTGRES
2026-08-11 06:26:04,446 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:26:04,447 [I] Checking connection
2026-08-11 06:26:04,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:26:04,673 [I] connection success
2026-08-11 06:26:04,673 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:26:04,673 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:26:04,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:26:04,904 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:26:04,905 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:26:04,905 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:26:04,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.4.148:40845/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:26:05,166 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:26:05.629522 7186 mini_ranger_kms.cc:310] Created kms service
I20260811 06:26:05.792143 7186 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:26:05.838225 7186 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:26:05 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429563, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:06.267406 7186 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:26:06.267776 7186 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms:/tmp/dist-test-task5M28Km/build/asan/bin/postgresql.jar:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5M28Km/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:26:06.267855 7186 mini_ranger_kms.cc:240] Using host: 127.7.4.148
I20260811 06:26:06.295928 7186 mini_ranger_kms.cc:276] Ranger KMS PID: 20700
I20260811 06:26:06.296059 7186 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.4.148:59101
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:26:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:26:08 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:08.720052 7186 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.506s user 0.068s sys 0.344s
I20260811 06:26:08.720314 7186 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:26:08.720494 7186 mini_ranger_kms.cc:401] 127.7.4.148:59101/kms/v1/keys
Aug 11 06:26:09 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
Aug 11 06:26:09 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.4.148@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
Aug 11 06:26:11 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:26:11.728176 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33837
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:33837
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:12.083688 20754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:12.084026 20754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:12.084192 20754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:12.091739 20754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:12.091887 20754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:12.091989 20754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:12.092104 20754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:12.092183 20754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:12.102495 20754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.4.148:59101/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:33837
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:12.103917 20754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:12.105250 20754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:12.118883 20760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:12.118911 20761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:12.120110 20764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:12.120200 20754 server_base.cc:1034] running on GCE node
I20260811 06:26:12.121007 20754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:12.122426 20754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:12.123654 20754 hybrid_clock.cc:648] HybridClock initialized: now 1786429572123600 us; error 48 us; skew 500 ppm
I20260811 06:26:12.124006 20754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:12 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:12.127342 20754 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:12.128988 20754 webserver.cc:460] Webserver started at http://127.7.4.190:39111/ using document root <none> and password file <none>
I20260811 06:26:12.129657 20754 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:12.129810 20754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:12.130136 20754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:26:12 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429572, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:12.184960 20754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "6ee36ace44674e2c92df9c2a6ae58027"
format_stamp: "Formatted at 2026-08-11 06:26:12 on dist-test-slave-07c2"
server_key: "457556a711fe2383188181c070cb63f9"
server_key_iv: "257159060e4f6da705533abc8545b9fb"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:12.185950 20754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "6ee36ace44674e2c92df9c2a6ae58027"
format_stamp: "Formatted at 2026-08-11 06:26:12 on dist-test-slave-07c2"
server_key: "457556a711fe2383188181c070cb63f9"
server_key_iv: "257159060e4f6da705533abc8545b9fb"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:12.191690 20754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:26:12.211443 20771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:12.213091 20754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:26:12.213306 20754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "6ee36ace44674e2c92df9c2a6ae58027"
format_stamp: "Formatted at 2026-08-11 06:26:12 on dist-test-slave-07c2"
server_key: "457556a711fe2383188181c070cb63f9"
server_key_iv: "257159060e4f6da705533abc8545b9fb"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:12.213549 20754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:12.238171 20754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:12.242174 20754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:12.242532 20754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:12.263200 20754 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:33837
I20260811 06:26:12.263221 20834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:33837 every 8 connection(s)
I20260811 06:26:12.264653 20754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:12.268620 20835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:12.272488 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 20754
I20260811 06:26:12.272735 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:26:12.273578 7186 external_mini_cluster.cc:1428] Setting key 6f5f7c8d3bd409a932ababea5ae149d3
I20260811 06:26:12.283005 20835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Bootstrap starting.
I20260811 06:26:12.288282 20835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:12.290005 20835 log.cc:824] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:12.293231 20835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: No bootstrap required, opened a new log
I20260811 06:26:12.302462 20835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:12.302991 20835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:12.303093 20835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee36ace44674e2c92df9c2a6ae58027, State: Initialized, Role: FOLLOWER
I20260811 06:26:12.303884 20835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:12.304174 20835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:12.304351 20835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:12.304519 20835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 06:26:12 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:12.306975 20835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:12.307634 20835 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee36ace44674e2c92df9c2a6ae58027; no voters:
I20260811 06:26:12.308399 20835 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:26:12.308553 20840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:12.312634 20840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 LEADER]: Becoming Leader. State: Replica: 6ee36ace44674e2c92df9c2a6ae58027, State: Running, Role: LEADER
I20260811 06:26:12.314040 20840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:12.315249 20835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:12.318342 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:12.275973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.7.4.190:33837)
0811 06:26:12.290826 (+ 14853us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:12.290842 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:12.290903 (+ 61us) server_negotiation.cc:407] Connection header received
0811 06:26:12.293476 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:12.293505 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:12.294059 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:12.294519 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:12.296288 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:12.298435 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:12.300433 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:12.301582 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:12.301759 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:12.306894 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:12.306906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:12.306926 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:12.306980 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:12.313004 (+ 6024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:12.313845 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:12.313855 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:12.313866 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:12.314044 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:12.314463 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:12.314469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:12.314472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:12.315220 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:12.315577 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:12.316205 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 06:26:12.316663 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14332,"thread_start_us":14290,"threads_started":1}
I20260811 06:26:12.329545 20841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee36ace44674e2c92df9c2a6ae58027. Latest consensus state: current_term: 1 leader_uuid: "6ee36ace44674e2c92df9c2a6ae58027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } } }
I20260811 06:26:12.329923 20841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:12.331441 20842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee36ace44674e2c92df9c2a6ae58027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } } }
I20260811 06:26:12.331802 20847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:12.331809 20842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:12.337405 20847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:12.358247 20847 catalog_manager.cc:1347] Generated new cluster ID: 2c0a6f7c9c794177bdf1c68ccf77f6b7
I20260811 06:26:12.358392 20847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:12.374668 20847 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:26:12.376699 20847 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:12.386519 20847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Generated new TSK 0
I20260811 06:26:12.387766 20847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:26:12.396396 20835 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:26:13.285137 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:13 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:26:13.723815 20884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:13.724073 20884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:13.724231 20884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:13.731395 20884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:13.731487 20884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:13.731652 20884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:13.746361 20884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:13.748299 20884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:13.750008 20884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:13.769398 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:13.769966 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:13.777117 20884 server_base.cc:1034] running on GCE node
W20260811 06:26:13.777393 20899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:13.778084 20884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:13.779124 20884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:13.780360 20884 hybrid_clock.cc:648] HybridClock initialized: now 1786429573780298 us; error 58 us; skew 500 ppm
I20260811 06:26:13.780738 20884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:13 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:13.783998 20884 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:13.785559 20884 webserver.cc:460] Webserver started at http://127.7.4.129:46361/ using document root <none> and password file <none>
I20260811 06:26:13.786216 20884 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:13.786319 20884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:13.786706 20884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:26:13 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:13.807436 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "d3e383e2c3234f788a8c3b9fb9b15230"
format_stamp: "Formatted at 2026-08-11 06:26:13 on dist-test-slave-07c2"
server_key: "49c4e14c29129b4b085b7697e122cf6f"
server_key_iv: "bc15eb40a5f71a9f46477593614ce72a"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:13.808596 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "d3e383e2c3234f788a8c3b9fb9b15230"
format_stamp: "Formatted at 2026-08-11 06:26:13 on dist-test-slave-07c2"
server_key: "49c4e14c29129b4b085b7697e122cf6f"
server_key_iv: "bc15eb40a5f71a9f46477593614ce72a"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:13.814481 20884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:26:13.833191 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:13.834894 20884 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:26:13.835153 20884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "d3e383e2c3234f788a8c3b9fb9b15230"
format_stamp: "Formatted at 2026-08-11 06:26:13 on dist-test-slave-07c2"
server_key: "49c4e14c29129b4b085b7697e122cf6f"
server_key_iv: "bc15eb40a5f71a9f46477593614ce72a"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:13.835480 20884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:13.929210 20884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:13.934473 20884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:13.934850 20884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:13.936112 20884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:13.937908 20884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:13.938050 20884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:13.938226 20884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:13.938349 20884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:26:13 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:14.184491 20884 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:34081
I20260811 06:26:14.184511 21030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:34081 every 8 connection(s)
I20260811 06:26:14.185886 20884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:14.186838 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 20884
I20260811 06:26:14.187107 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:26:14.188176 7186 external_mini_cluster.cc:1428] Setting key 63eecb660338b16122715cbdcb08e545
Aug 11 06:26:14 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
I20260811 06:26:14.246766 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:14.206159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.129:58681)
0811 06:26:14.218515 (+ 12356us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:14.218576 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:14.222019 (+ 3443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:14.226225 (+ 4206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:14.226238 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:14.226858 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:14.227697 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:14.227722 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:14.229301 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:14.229307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:14.231353 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:14.231369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:14.232489 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:14.232496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:14.232830 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:14.233633 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:14.233665 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:14.236479 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:14.241748 (+ 5269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:14.241756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:14.241772 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:14.242298 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:14.242774 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:14.242778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:14.242782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:14.243040 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:14.243825 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:14.243835 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:14.244312 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 06:26:14.244642 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 06:26:14.245191 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10824,"thread_start_us":11022,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
I20260811 06:26:14.248339 21035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:14.205297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:58681 (local address 127.7.4.190:33837)
0811 06:26:14.225463 (+ 20166us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:14.225470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:14.225491 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:26:14.225558 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:14.225563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:14.225661 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:14.227373 (+ 1712us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:14.227913 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:14.229110 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:14.231551 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:14.232228 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:14.233277 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:14.236740 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:14.236746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:14.236751 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:14.236790 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:14.240542 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:14.242470 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:14.242475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:14.242478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:14.242627 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:14.243199 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:14.243204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:14.243207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:14.243630 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:14.247610 (+ 3980us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:14.247752 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 06:26:14.247972 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20022,"thread_start_us":20453,"threads_started":1}
I20260811 06:26:14.249897 21031 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:14.250197 21031 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:14.250970 21031 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:14.253345 20792 ts_manager.cc:194] Registered new tserver with Master: d3e383e2c3234f788a8c3b9fb9b15230 (127.7.4.129:34081)
I20260811 06:26:14.255244 20792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:58681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:26:14.288494 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:14 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429573, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
W20260811 06:26:14.762861 21040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:14.763231 21040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:14.763439 21040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:14.774564 21040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:14.774711 21040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:14.774950 21040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:14.790179 21040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:14.791908 21040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:14.793584 21040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:14.809028 21055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:14.810568 21056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:14.821206 21058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:14.824868 21040 server_base.cc:1034] running on GCE node
I20260811 06:26:14.825616 21040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:14.826829 21040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:14.828179 21040 hybrid_clock.cc:648] HybridClock initialized: now 1786429574828057 us; error 104 us; skew 500 ppm
I20260811 06:26:14.828604 21040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:14 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:14.833037 21040 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:14.835155 21040 webserver.cc:460] Webserver started at http://127.7.4.130:39849/ using document root <none> and password file <none>
I20260811 06:26:14.836083 21040 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:14.836225 21040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:14.836545 21040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:26:14 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:14.864849 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "0d321545cba84d2a934388a91a3e449b"
format_stamp: "Formatted at 2026-08-11 06:26:14 on dist-test-slave-07c2"
server_key: "cf25507d0a712decce056016e0021b5e"
server_key_iv: "12109eb615452cae2435b41588a6a8d1"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:14.865909 21040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "0d321545cba84d2a934388a91a3e449b"
format_stamp: "Formatted at 2026-08-11 06:26:14 on dist-test-slave-07c2"
server_key: "cf25507d0a712decce056016e0021b5e"
server_key_iv: "12109eb615452cae2435b41588a6a8d1"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:14.893380 21040 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.012s sys 0.016s
I20260811 06:26:14.924265 21065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:14.926141 21040 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.008s
I20260811 06:26:14.926353 21040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "0d321545cba84d2a934388a91a3e449b"
format_stamp: "Formatted at 2026-08-11 06:26:14 on dist-test-slave-07c2"
server_key: "cf25507d0a712decce056016e0021b5e"
server_key_iv: "12109eb615452cae2435b41588a6a8d1"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:14.926640 21040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:14.972839 21040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:14.977195 21040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:14.977525 21040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:14.978615 21040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:14.980203 21040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:14.980302 21040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:14.980463 21040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:14.980537 21040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:15.023614 21040 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:42943
I20260811 06:26:15.023650 21187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:42943 every 8 connection(s)
I20260811 06:26:15.024864 21040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:15.033171 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21040
I20260811 06:26:15.033547 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:26:15.034216 7186 external_mini_cluster.cc:1428] Setting key e50f7a57205b07c6e42f4a3cca283174
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429574, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:15.050339 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.027623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:52615 (local address 127.7.4.190:33837)
0811 06:26:15.028161 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:15.028167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:15.029242 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 06:26:15.031877 (+ 2635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:15.031881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:15.031994 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:15.032139 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:15.034196 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.035178 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.037387 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.037998 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.038127 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.042720 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:15.042729 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:15.042736 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:15.042780 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:15.046644 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.047495 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.047503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.047509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.047663 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.048176 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.048183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.048189 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.048615 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:15.048767 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:15.049618 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 06:26:15.049905 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":246,"threads_started":1}
I20260811 06:26:15.051242 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.028141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.130:52615)
0811 06:26:15.029041 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:15.029105 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:15.031557 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:15.032473 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:15.032482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:15.033190 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:15.034024 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:15.034045 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.035393 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.035396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:15.037224 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:15.037241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.038242 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.038252 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:15.038583 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.039424 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:15.039458 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:15.042464 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:15.046797 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:15.046805 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:15.046820 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:15.047366 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:15.047779 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:15.047784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:15.047787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:15.048056 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:15.048753 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:15.048760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:15.049132 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 06:26:15.049454 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 06:26:15.049971 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":367,"threads_started":1}
I20260811 06:26:15.052747 21188 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:15.053063 21188 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:15.053926 21188 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:15.055598 20792 ts_manager.cc:194] Registered new tserver with Master: 0d321545cba84d2a934388a91a3e449b (127.7.4.130:42943)
I20260811 06:26:15.056597 20792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:52615
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:26:15.102435 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:15.258663 21031 heartbeater.cc:503] Master 127.7.4.190:33837 was elected leader, sending a full tablet report...
W20260811 06:26:15.460218 21194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:15.460618 21194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:15.460734 21194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:15.468137 21194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:15.468236 21194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:15.468348 21194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:15.478206 21194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:15.479468 21194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:15.480772 21194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:15.494086 21202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:15.494191 21200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:15.494171 21199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:15.495431 21194 server_base.cc:1034] running on GCE node
I20260811 06:26:15.496052 21194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:15.497004 21194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:15.498224 21194 hybrid_clock.cc:648] HybridClock initialized: now 1786429575498190 us; error 50 us; skew 500 ppm
I20260811 06:26:15.498550 21194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:15.501735 21194 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:15.503182 21194 webserver.cc:460] Webserver started at http://127.7.4.131:38297/ using document root <none> and password file <none>
I20260811 06:26:15.503885 21194 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:15.503994 21194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:15.504251 21194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:15.519177 21194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "ab0d6dd16f3742488c6a1a703ac14c93"
format_stamp: "Formatted at 2026-08-11 06:26:15 on dist-test-slave-07c2"
server_key: "c1e2219d74af230c0c9d368ecf73bbcd"
server_key_iv: "ba67dc5e702bf991abcd897d2075fb52"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:15.520313 21194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "ab0d6dd16f3742488c6a1a703ac14c93"
format_stamp: "Formatted at 2026-08-11 06:26:15 on dist-test-slave-07c2"
server_key: "c1e2219d74af230c0c9d368ecf73bbcd"
server_key_iv: "ba67dc5e702bf991abcd897d2075fb52"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:15.527275 21194 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 06:26:15.544229 21209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:15.545917 21194 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 06:26:15.546135 21194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "ab0d6dd16f3742488c6a1a703ac14c93"
format_stamp: "Formatted at 2026-08-11 06:26:15 on dist-test-slave-07c2"
server_key: "c1e2219d74af230c0c9d368ecf73bbcd"
server_key_iv: "ba67dc5e702bf991abcd897d2075fb52"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:15.546384 21194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:15.567829 21194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:15.572175 21194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:15.572528 21194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:15.573940 21194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:15.576052 21194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:15.576174 21194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:15.576333 21194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:15.576407 21194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:15.614565 21194 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:39431
I20260811 06:26:15.614611 21331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:39431 every 8 connection(s)
I20260811 06:26:15.616065 21194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:15.617890 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21194
I20260811 06:26:15.618088 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:26:15.618669 7186 external_mini_cluster.cc:1428] Setting key ebc80bb75e85092626b71ca4e55991e7
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429575, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:15.638885 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.619049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:46827 (local address 127.7.4.190:33837)
0811 06:26:15.619611 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:15.619619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:15.620848 (+ 1229us) server_negotiation.cc:407] Connection header received
0811 06:26:15.623277 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:15.623283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:15.623431 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:15.623554 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:15.625034 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.625770 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.627377 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.627851 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.627987 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.631866 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:15.631871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:15.631874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:15.631910 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:15.635422 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.636377 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.636382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.636392 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.636525 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.637040 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.637044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.637047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.637340 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:15.637500 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:15.638256 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 06:26:15.638504 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":288,"threads_started":1}
I20260811 06:26:15.639946 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.619729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.131:46827)
0811 06:26:15.620673 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:15.620723 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:15.622989 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:15.623819 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:15.623827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:15.624273 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:15.624859 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:15.624876 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.625942 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.625948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:15.627208 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:15.627221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.628097 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.628105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:15.628352 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.629029 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:15.629052 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:15.631630 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:15.635632 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:15.635640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:15.635655 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:15.636251 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:15.636647 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:15.636652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:15.636656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:15.636906 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:15.637474 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:15.637482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:15.637854 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 06:26:15.638136 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:26:15.638701 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":372,"threads_started":1}
I20260811 06:26:15.641513 21333 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:15.641844 21333 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:15.642639 21333 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:15.644129 20792 ts_manager.cc:194] Registered new tserver with Master: ab0d6dd16f3742488c6a1a703ac14c93 (127.7.4.131:39431)
I20260811 06:26:15.645001 20792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:46827
I20260811 06:26:15.647516 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:26:15.649273 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 20884
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:26:15.666465 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.650370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.7.4.129:34081)
0811 06:26:15.651229 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:15.651235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:15.651262 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:26:15.651376 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:15.651385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:15.651621 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:15.651742 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:15.653228 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.654262 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.655878 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.656412 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.656513 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.660894 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:15.660906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:15.660920 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:15.660958 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:15.663780 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.664463 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.664469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.664472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.664620 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.664984 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.664987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.664989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.665307 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:15.665541 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:15.665798 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:26:15.666040 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":463,"threads_started":1}
I20260811 06:26:15.667857 21024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:15.751636 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 20884
I20260811 06:26:15.772632 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21040
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:26:15.788514 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.773480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.7.4.130:42943)
0811 06:26:15.774087 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:15.774094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:15.774120 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:26:15.774182 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:15.774189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:15.774402 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:15.774536 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:15.775866 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.776917 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.778196 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.778735 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.778829 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.783139 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:15.783152 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:15.783166 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:15.783198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:15.785852 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.786524 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.786531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.786533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.786675 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.786997 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.787000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.787002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.787397 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:15.787628 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:15.787996 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 06:26:15.788221 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":331,"threads_started":1}
I20260811 06:26:15.789758 21183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:15.872516 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21040
I20260811 06:26:15.897279 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21194
Aug 11 06:26:15 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:26:15.913128 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:15.898356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.7.4.131:39431)
0811 06:26:15.899027 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:15.899034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:15.899063 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:26:15.899121 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:15.899128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:15.899415 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:15.899546 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:15.900821 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.901881 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.903109 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:15.903800 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:15.903898 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:15.908082 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:15.908093 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:15.908106 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:15.908138 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:15.910721 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.911399 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.911406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.911408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.911531 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:15.911858 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:15.911861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:15.911863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:15.912172 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:15.912421 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:15.912672 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 06:26:15.912856 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":375,"threads_started":1}
I20260811 06:26:15.914341 21327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:16.001113 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21194
I20260811 06:26:16.020853 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 20754
I20260811 06:26:16.022125 20830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:16.130040 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 20754
I20260811 06:26:16.157130 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33837
--webserver_interface=127.7.4.190
--webserver_port=39111
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:33837
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:16.486474 21345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:16.486797 21345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:16.486953 21345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:16.494030 21345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:16.494176 21345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:16.494275 21345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:16.494369 21345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:16.494447 21345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:16.504186 21345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:33837
--ranger_config_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=39111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:16.505378 21345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:16.506642 21345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:16.518394 21354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:16.518546 21350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:16.518615 21352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:16.519618 21345 server_base.cc:1034] running on GCE node
I20260811 06:26:16.520431 21345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:16.521741 21345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:16.522971 21345 hybrid_clock.cc:648] HybridClock initialized: now 1786429576522937 us; error 47 us; skew 500 ppm
I20260811 06:26:16.523427 21345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:16 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:16.530617 21345 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:16.532135 21345 webserver.cc:460] Webserver started at http://127.7.4.190:39111/ using document root <none> and password file <none>
I20260811 06:26:16.532829 21345 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:16.532995 21345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:26:16 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429576, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:16.614631 21345 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:26:16.617851 21362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:16.619385 21345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:26:16.619585 21345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "6ee36ace44674e2c92df9c2a6ae58027"
format_stamp: "Formatted at 2026-08-11 06:26:12 on dist-test-slave-07c2"
server_key: "457556a711fe2383188181c070cb63f9"
server_key_iv: "257159060e4f6da705533abc8545b9fb"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:16.620556 21345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:16.644390 21345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:16.648360 21345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:16.648670 21345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:16.669138 21345 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:33837
I20260811 06:26:16.669167 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:33837 every 8 connection(s)
I20260811 06:26:16.670589 21345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:16.675203 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21345
I20260811 06:26:16.676316 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:34081
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:33837
--webserver_port=46361
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:16.677763 21420 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:26:16.681123 21420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Bootstrap starting.
I20260811 06:26:16.704212 21420 log.cc:824] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:16.716419 21420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:26:16.717029 21420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Bootstrap complete.
I20260811 06:26:16.730798 21420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:16.732053 21420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee36ace44674e2c92df9c2a6ae58027, State: Initialized, Role: FOLLOWER
I20260811 06:26:16.732779 21420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:16.733038 21420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:16.733213 21420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:16.733405 21420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:26:16.738852 21420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:16.739450 21420 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee36ace44674e2c92df9c2a6ae58027; no voters:
I20260811 06:26:16.740114 21420 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:26:16.740276 21425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:26:16.747308 21425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [term 2 LEADER]: Becoming Leader. State: Replica: 6ee36ace44674e2c92df9c2a6ae58027, State: Running, Role: LEADER
I20260811 06:26:16.747915 21425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } }
I20260811 06:26:16.748845 21420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:16.760504 21427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee36ace44674e2c92df9c2a6ae58027. Latest consensus state: current_term: 2 leader_uuid: "6ee36ace44674e2c92df9c2a6ae58027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } } }
I20260811 06:26:16.761200 21427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:16.766489 21426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ee36ace44674e2c92df9c2a6ae58027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee36ace44674e2c92df9c2a6ae58027" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33837 } } }
I20260811 06:26:16.767473 21426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:16.774286 21433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:16.788858 21433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:16.790226 21433 catalog_manager.cc:1259] Loaded cluster ID: 2c0a6f7c9c794177bdf1c68ccf77f6b7
I20260811 06:26:16.790388 21433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:16.798113 21433 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:16.804526 21433 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Loaded TSK: 0
I20260811 06:26:16.806207 21433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:26:16.819914 21420 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 06:26:17.188936 21423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:17.189476 21423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:17.189690 21423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:17.201436 21423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:17.201639 21423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:17.201886 21423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:17.217509 21423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:34081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=46361
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:17.218997 21423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:17.220832 21423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:17.255811 21462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:17.269047 21467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:17.269085 21423 server_base.cc:1034] running on GCE node
W20260811 06:26:17.269296 21463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:17.270278 21423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:17.271888 21423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:17.273162 21423 hybrid_clock.cc:648] HybridClock initialized: now 1786429577273088 us; error 63 us; skew 500 ppm
I20260811 06:26:17.273591 21423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:17 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:17.277863 21423 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:17.291213 21423 webserver.cc:460] Webserver started at http://127.7.4.129:46361/ using document root <none> and password file <none>
I20260811 06:26:17.292112 21423 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:17.292292 21423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:26:17 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:17.332674 21423 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.004s sys 0.008s
I20260811 06:26:17.340564 21475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:17.342338 21423 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.004s
I20260811 06:26:17.342545 21423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "d3e383e2c3234f788a8c3b9fb9b15230"
format_stamp: "Formatted at 2026-08-11 06:26:13 on dist-test-slave-07c2"
server_key: "49c4e14c29129b4b085b7697e122cf6f"
server_key_iv: "bc15eb40a5f71a9f46477593614ce72a"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:17.344090 21423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:17.365818 21423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:17.371083 21423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:17.371456 21423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:17.372795 21423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:17.374778 21423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:17.374900 21423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:17.375025 21423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:17.375118 21423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:17.443058 21423 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:34081
I20260811 06:26:17.443058 21599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:34081 every 8 connection(s)
I20260811 06:26:17.444820 21423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:17.450865 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21423
I20260811 06:26:17.451877 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:42943
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:33837
--webserver_port=39849
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:17 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:17.486562 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:17.448402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44559 (local address 127.7.4.190:33837)
0811 06:26:17.466590 (+ 18188us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:17.466608 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:17.466658 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:26:17.467041 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:17.467059 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:17.467575 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:17.467944 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:17.470460 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:17.472336 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:17.474275 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:17.474923 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:17.475149 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:17.478637 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:17.478645 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:17.478659 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:17.478698 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:17.481839 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:17.482558 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:17.482565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:17.482570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:17.482727 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:17.483161 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:17.483167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:17.483170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:17.483805 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:17.484029 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:17.484857 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 06:26:17.485217 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17884,"spinlock_wait_cycles":2432,"thread_start_us":17782,"threads_started":1}
I20260811 06:26:17.486596 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:17.448358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.129:44559)
0811 06:26:17.462774 (+ 14416us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:17.462836 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:17.465596 (+ 2760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:17.468699 (+ 3103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:17.468711 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:17.469406 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:17.470222 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:17.470250 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:17.472581 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:17.472587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:17.474120 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:17.474133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:17.475187 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:17.475195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:17.475488 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:17.476242 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:17.476270 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:17.478380 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:17.481979 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:17.481988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:17.482003 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:17.482408 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:17.482857 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:17.482862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:17.482865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:17.483050 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:17.483969 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:17.483977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:17.484317 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 06:26:17.484657 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 06:26:17.485199 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14030,"thread_start_us":20368,"threads_started":1}
I20260811 06:26:17.490031 21600 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:17.490351 21600 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:17.491044 21600 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
W20260811 06:26:17.494601 21600 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:33837 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:26:17.494855 21600 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:17.495157 21600 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:17.496276 21600 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:17.496551 21600 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
W20260811 06:26:17.497563 21600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:26:17.905443 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:17.905879 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:17.906080 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:17.917904 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:17.918059 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:17.918305 21604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:17.934527 21604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:42943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=39849
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:17.936494 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:17.938241 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:17.956620 21619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:17.956944 21620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:17.960139 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:17.961030 21604 server_base.cc:1034] running on GCE node
I20260811 06:26:17.961675 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:17.962790 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:17.964038 21604 hybrid_clock.cc:648] HybridClock initialized: now 1786429577963975 us; error 47 us; skew 500 ppm
I20260811 06:26:17.964447 21604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:17 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:17.968726 21604 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:17.970646 21604 webserver.cc:460] Webserver started at http://127.7.4.130:39849/ using document root <none> and password file <none>
I20260811 06:26:17.971719 21604 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:17.971884 21604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:26:17 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:17.998586 21604 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260811 06:26:18.002718 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.004685 21604 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 06:26:18.004895 21604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "0d321545cba84d2a934388a91a3e449b"
format_stamp: "Formatted at 2026-08-11 06:26:14 on dist-test-slave-07c2"
server_key: "cf25507d0a712decce056016e0021b5e"
server_key_iv: "12109eb615452cae2435b41588a6a8d1"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:18.006423 21604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:18.048903 21604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:18.054646 21604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:18.055027 21604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:18.056416 21604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:18.058279 21604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:18.058403 21604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.058518 21604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:18.058597 21604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.176474 21767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:42943 every 8 connection(s)
I20260811 06:26:18.176874 21604 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:42943
I20260811 06:26:18.178859 21604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:18.184695 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21604
I20260811 06:26:18.185611 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.4.148:59101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:39431
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:33837
--webserver_port=38297
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429577, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:18.252570 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:18.188363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:49633 (local address 127.7.4.190:33837)
0811 06:26:18.216532 (+ 28169us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:18.216559 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:18.222100 (+ 5541us) server_negotiation.cc:407] Connection header received
0811 06:26:18.222225 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:18.222234 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:18.222367 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:18.222501 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:18.234188 (+ 11687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.235456 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.238079 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.238758 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.238864 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:18.244210 (+ 5346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:18.244218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:18.244223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:18.244271 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:18.248140 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.249324 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.249331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.249335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.249503 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.250077 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.250083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.250086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.250617 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:18.250764 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:18.251930 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0811 06:26:18.252187 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27941,"thread_start_us":28215,"threads_started":1}
I20260811 06:26:18.254909 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:18.183760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.130:49633)
0811 06:26:18.217564 (+ 33804us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:18.217657 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:18.221395 (+ 3738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:18.232095 (+ 10700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:18.232110 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:18.232841 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:18.233840 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:18.233876 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.235704 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.235709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:18.237846 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:18.237876 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.239257 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.239264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:18.239593 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:18.240519 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:18.240553 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:18.243870 (+ 3317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:18.248350 (+ 4480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:18.248360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:18.248379 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:18.249073 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:18.249661 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:18.249666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:18.249670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:18.249924 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:18.250873 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:18.250881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:18.251454 (+ 573us) client_negotiation.cc:769] Sending connection context
0811 06:26:18.252882 (+ 1428us) client_negotiation.cc:240] Negotiation successful
0811 06:26:18.253498 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33306,"thread_start_us":38825,"threads_started":1}
I20260811 06:26:18.256896 21768 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:18.257249 21768 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:18.258144 21768 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:18.260375 21380 ts_manager.cc:194] Registered new tserver with Master: 0d321545cba84d2a934388a91a3e449b (127.7.4.130:42943)
I20260811 06:26:18.262603 21380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:49633
I20260811 06:26:18.497746 21600 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:18.498236 21600 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:18.499752 21380 ts_manager.cc:194] Registered new tserver with Master: d3e383e2c3234f788a8c3b9fb9b15230 (127.7.4.129:34081)
I20260811 06:26:18.500973 21380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:44559
W20260811 06:26:18.671454 21772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:18.671870 21772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:18.672104 21772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:18.683506 21772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:18.683691 21772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:18.683928 21772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:18.696434 21772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.4.148:59101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:39431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=38297
--tserver_master_addrs=127.7.4.190:33837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:18.697979 21772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:18.699486 21772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
W20260811 06:26:18.753357 21779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:18.761011 21778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:18.762842 21781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:18.763899 21772 server_base.cc:1034] running on GCE node
I20260811 06:26:18.764883 21772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:18.772588 21772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:18.776223 21772 hybrid_clock.cc:648] HybridClock initialized: now 1786429578776155 us; error 67 us; skew 500 ppm
I20260811 06:26:18.776678 21772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:18.780985 21772 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:18.807833 21772 webserver.cc:460] Webserver started at http://127.7.4.131:38297/ using document root <none> and password file <none>
I20260811 06:26:18.808683 21772 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:18.808895 21772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:18.846123 21772 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:26:18.849023 21791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.850793 21772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:26:18.850979 21772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "ab0d6dd16f3742488c6a1a703ac14c93"
format_stamp: "Formatted at 2026-08-11 06:26:15 on dist-test-slave-07c2"
server_key: "c1e2219d74af230c0c9d368ecf73bbcd"
server_key_iv: "ba67dc5e702bf991abcd897d2075fb52"
server_key_version: "kuduclusterkey@0"
I20260811 06:26:18.852068 21772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:18.870817 21772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:18.874794 21772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:18.875064 21772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:18.876209 21772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:18.877686 21772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:18.877816 21772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.877951 21772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:18.878047 21772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:18.921700 21772 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:39431
I20260811 06:26:18.921752 21919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:39431 every 8 connection(s)
I20260811 06:26:18.923136 21772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:18.924974 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 21772
Aug 11 06:26:18 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:18.975041 21925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:18.926307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.7.4.190:33837)
0811 06:26:18.954374 (+ 28067us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:18.954396 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:18.954426 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:26:18.954518 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:18.954526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:18.954645 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:18.954814 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:18.958717 (+ 3903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.959983 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.962196 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.962795 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.962931 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:18.966934 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:18.966940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:18.966947 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:18.966985 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:18.971009 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.971953 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.971958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.971961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.972127 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.972592 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.972599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.972607 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.972963 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:18.973511 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:18.974499 (+ 988us) server_negotiation.cc:299] Negotiation successful
0811 06:26:18.974749 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27854,"thread_start_us":40986,"threads_started":1}
I20260811 06:26:18.981631 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:18.934861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:33837 (local address 127.7.4.131:44257)
0811 06:26:18.958213 (+ 23352us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:18.958269 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:18.960660 (+ 2391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:18.961671 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:18.961684 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:18.962310 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:18.963106 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:18.963132 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.964775 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.964781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:18.967024 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:18.967040 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.968503 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.968513 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:18.968828 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:18.969491 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:18.969524 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:18.972343 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:18.977009 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:18.977017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:18.977033 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:18.977601 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:18.978029 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:18.978035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:18.978039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:18.978275 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:18.978997 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:18.979004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:18.979397 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 06:26:18.979717 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 06:26:18.980233 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23050,"thread_start_us":22834,"threads_started":1}
I20260811 06:26:18.982849 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:18.934142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:44257 (local address 127.7.4.190:33837)
0811 06:26:18.959728 (+ 25586us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:18.959734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:18.959755 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:26:18.961051 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:18.961056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:18.961132 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:18.961231 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:18.963363 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.964582 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.967212 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:18.968040 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:18.968164 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:18.972630 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:18.972636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:18.972641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:18.972678 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:18.976847 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.977750 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.977755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.977758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.977890 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:18.978421 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:18.978426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:18.978429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:18.978759 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:18.978899 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:18.982180 (+ 3281us) server_negotiation.cc:299] Negotiation successful
0811 06:26:18.982366 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25443,"thread_start_us":19881,"threads_started":1}
I20260811 06:26:18.984730 21921 heartbeater.cc:347] Connected to a master server at 127.7.4.190:33837
I20260811 06:26:18.985052 21921 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:18.985773 21921 heartbeater.cc:511] Master 127.7.4.190:33837 requested a full tablet report, sending...
I20260811 06:26:18.987371 21380 ts_manager.cc:194] Registered new tserver with Master: ab0d6dd16f3742488c6a1a703ac14c93 (127.7.4.131:39431)
I20260811 06:26:18.988184 21380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:44257
I20260811 06:26:18.993484 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:26:19.022032 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.002783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.7.4.190:33837)
0811 06:26:19.002952 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.002958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.005593 (+ 2635us) server_negotiation.cc:407] Connection header received
0811 06:26:19.005733 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.005739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.005817 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.005930 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:19.007830 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.008895 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.010778 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.011430 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.011530 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.014943 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:19.014950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:19.014955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:19.014994 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:19.018715 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.019739 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.019745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.019748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.019905 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.020410 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.020414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.020416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.020692 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:19.020941 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.021371 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.021677 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 06:26:19 dist-test-slave-07c2 krb5kdc[20268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429578, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for HTTP/127.7.4.148@KRBTEST.COM
I20260811 06:26:19.028227 21380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:26:19.043427 21380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:26:19.149842 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.114477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.7.4.129:34081)
0811 06:26:19.120027 (+ 5550us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.120040 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.135484 (+ 15444us) server_negotiation.cc:407] Connection header received
0811 06:26:19.135927 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.135937 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.136210 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.136414 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:19.137482 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.139025 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.146662 (+ 7637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.148138 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.148236 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.148490 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.149303 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.149484 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5335,"thread_start_us":5186,"threads_started":1}
I20260811 06:26:19.150681 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.114056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:34081 (local address 127.0.0.1:53526)
0811 06:26:19.135343 (+ 21287us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:19.135373 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:19.135520 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:19.136763 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:19.136770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:19.136804 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:19.137278 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.137292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.139228 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.139233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.141631 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.141646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.148785 (+ 7139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.148793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.149007 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.149067 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 06:26:19.150137 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0811 06:26:19.150349 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21126,"thread_start_us":34898,"threads_started":1}
I20260811 06:26:19.156288 21525 tablet_service.cc:1467] Processing CreateTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 (DEFAULT_TABLE table=test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:19.158353 21525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e254a87888f14f8ca3c9c9f285ea4c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:19.160493 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.117699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:39431 (local address 127.0.0.1:51466)
0811 06:26:19.152284 (+ 34585us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:19.152303 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:19.152414 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:19.153435 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:19.153439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:19.153465 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:19.153619 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.153630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.155592 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.155598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.157530 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.157574 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.159738 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.159744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.159918 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.159951 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:26:19.160041 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:26:19.160201 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34480,"thread_start_us":17911,"threads_started":1}
I20260811 06:26:19.161301 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.117968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.7.4.131:39431)
0811 06:26:19.148221 (+ 30253us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.148234 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.152547 (+ 4313us) server_negotiation.cc:407] Connection header received
0811 06:26:19.152654 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.152661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.152886 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.153084 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:19.153807 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.155435 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.157738 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.159219 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.159432 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.159651 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.160806 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.160965 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30000,"thread_start_us":30282,"threads_started":1}
I20260811 06:26:19.165187 21843 tablet_service.cc:1467] Processing CreateTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 (DEFAULT_TABLE table=test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:19.168273 21843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e254a87888f14f8ca3c9c9f285ea4c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:19.187089 21946 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Bootstrap starting.
I20260811 06:26:19.190335 21946 tablet_bootstrap.cc:654] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:19.191394 21946 log.cc:824] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:19.198896 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.113831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:42943 (local address 127.0.0.1:37202)
0811 06:26:19.121124 (+ 7293us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:19.121169 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:19.121360 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:19.183679 (+ 62319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:19.183688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:19.183737 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:19.184204 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.184221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.191375 (+ 7154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.191381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.193500 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.193519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.198088 (+ 4569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.198095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.198278 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.198319 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:26:19.198398 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 06:26:19.198546 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7048,"thread_start_us":6875,"threads_started":1}
I20260811 06:26:19.200438 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.114063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.7.4.130:42943)
0811 06:26:19.149353 (+ 35290us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.149363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.149396 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:26:19.149495 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.149503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.149755 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.183669 (+ 33914us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:19.186066 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.191148 (+ 5082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.193870 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.196851 (+ 2981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.199535 (+ 2684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.199782 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.199857 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.200062 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35127,"thread_start_us":35123,"threads_started":1}
I20260811 06:26:19.210559 21690 tablet_service.cc:1467] Processing CreateTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 (DEFAULT_TABLE table=test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:19.212481 21690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e254a87888f14f8ca3c9c9f285ea4c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:19.225005 21948 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Bootstrap starting.
I20260811 06:26:19.231326 21946 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: No bootstrap required, opened a new log
I20260811 06:26:19.231632 21946 ts_tablet_manager.cc:1397] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Time spent bootstrapping tablet: real 0.045s user 0.017s sys 0.027s
I20260811 06:26:19.236253 21948 tablet_bootstrap.cc:654] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:19.241966 21948 log.cc:824] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:19.242223 21946 raft_consensus.cc:348] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.242659 21946 raft_consensus.cc:374] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:19.242762 21946 raft_consensus.cc:729] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e383e2c3234f788a8c3b9fb9b15230, State: Initialized, Role: FOLLOWER
I20260811 06:26:19.243534 21946 consensus_queue.cc:260] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.254196 21948 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: No bootstrap required, opened a new log
I20260811 06:26:19.255173 21948 ts_tablet_manager.cc:1397] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.003s
I20260811 06:26:19.255486 21949 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Bootstrap starting.
I20260811 06:26:19.260849 21949 tablet_bootstrap.cc:654] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:19.266923 21600 heartbeater.cc:503] Master 127.7.4.190:33837 was elected leader, sending a full tablet report...
I20260811 06:26:19.267673 21946 ts_tablet_manager.cc:1428] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Time spent starting tablet: real 0.036s user 0.019s sys 0.017s
I20260811 06:26:19.268390 21768 heartbeater.cc:503] Master 127.7.4.190:33837 was elected leader, sending a full tablet report...
I20260811 06:26:19.272835 21949 log.cc:824] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:19.282339 21949 tablet_bootstrap.cc:492] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: No bootstrap required, opened a new log
I20260811 06:26:19.282672 21949 ts_tablet_manager.cc:1397] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.005s
I20260811 06:26:19.283236 21948 raft_consensus.cc:348] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.283986 21948 raft_consensus.cc:374] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:19.284123 21948 raft_consensus.cc:729] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0d6dd16f3742488c6a1a703ac14c93, State: Initialized, Role: FOLLOWER
I20260811 06:26:19.284757 21948 consensus_queue.cc:260] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.292021 21921 heartbeater.cc:503] Master 127.7.4.190:33837 was elected leader, sending a full tablet report...
I20260811 06:26:19.293116 21948 ts_tablet_manager.cc:1428] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Time spent starting tablet: real 0.032s user 0.010s sys 0.009s
I20260811 06:26:19.296413 21949 raft_consensus.cc:348] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.296772 21949 raft_consensus.cc:374] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:19.296860 21949 raft_consensus.cc:729] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d321545cba84d2a934388a91a3e449b, State: Initialized, Role: FOLLOWER
I20260811 06:26:19.297464 21949 consensus_queue.cc:260] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.299194 21949 ts_tablet_manager.cc:1428] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Time spent starting tablet: real 0.016s user 0.012s sys 0.001s
W20260811 06:26:19.435611 21922 tablet.cc:2369] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:26:19.448215 21601 tablet.cc:2369] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:19.482331 21954 raft_consensus.cc:482] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:19.482668 21954 raft_consensus.cc:504] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
W20260811 06:26:19.483359 21769 tablet.cc:2369] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:19.484872 21954 leader_election.cc:288] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e383e2c3234f788a8c3b9fb9b15230 (127.7.4.129:34081), ab0d6dd16f3742488c6a1a703ac14c93 (127.7.4.131:39431)
I20260811 06:26:19.491743 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.485447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38149 (local address 127.7.4.131:39431)
0811 06:26:19.485715 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.485720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.486361 (+ 641us) server_negotiation.cc:407] Connection header received
0811 06:26:19.486568 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.486572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.486657 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.486768 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:19.487510 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.488190 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.490015 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.490754 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.490905 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.491021 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.491345 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.491472 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 06:26:19.491757 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.485186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:50801 (local address 127.7.4.129:34081)
0811 06:26:19.485431 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.485438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.486244 (+ 806us) server_negotiation.cc:407] Connection header received
0811 06:26:19.486471 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.486475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.486547 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.486668 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:19.487377 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.488088 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.489983 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.490758 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.490918 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.491032 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.491380 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.491502 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:26:19.491964 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.485032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:34081 (local address 127.7.4.130:50801)
0811 06:26:19.486118 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:19.486138 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:19.486318 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:19.486706 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:19.486710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:19.486745 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:19.487198 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.487211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.488233 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.488238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.489832 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.489844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.491038 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.491046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.491189 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.491211 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:26:19.491339 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 06:26:19.491466 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":48,"spinlock_wait_cycles":21376,"thread_start_us":651,"threads_started":1}
I20260811 06:26:19.492250 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.485338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:39431 (local address 127.7.4.130:38149)
0811 06:26:19.486254 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:19.486269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:19.486421 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:19.486781 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:19.486786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:19.486859 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:19.487343 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.487355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.488334 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.488339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.489863 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:19.489875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.490985 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.490992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:19.491162 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.491190 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 06:26:19.491320 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 06:26:19.491447 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":8,"thread_start_us":575,"threads_started":1}
I20260811 06:26:19.492933 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e254a87888f14f8ca3c9c9f285ea4c50" candidate_uuid: "0d321545cba84d2a934388a91a3e449b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" is_pre_election: true
I20260811 06:26:19.493234 21868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e254a87888f14f8ca3c9c9f285ea4c50" candidate_uuid: "0d321545cba84d2a934388a91a3e449b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" is_pre_election: true
I20260811 06:26:19.493508 21546 raft_consensus.cc:2454] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d321545cba84d2a934388a91a3e449b in term 0.
I20260811 06:26:19.493734 21868 raft_consensus.cc:2454] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d321545cba84d2a934388a91a3e449b in term 0.
I20260811 06:26:19.494196 21639 leader_election.cc:302] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d321545cba84d2a934388a91a3e449b, d3e383e2c3234f788a8c3b9fb9b15230; no voters:
I20260811 06:26:19.494571 21954 raft_consensus.cc:2781] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:26:19.494748 21954 raft_consensus.cc:482] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:26:19.494881 21954 raft_consensus.cc:3037] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:19.497085 21954 raft_consensus.cc:504] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.497644 21954 leader_election.cc:288] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [CANDIDATE]: Term 1 election: Requested vote from peers d3e383e2c3234f788a8c3b9fb9b15230 (127.7.4.129:34081), ab0d6dd16f3742488c6a1a703ac14c93 (127.7.4.131:39431)
I20260811 06:26:19.498096 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e254a87888f14f8ca3c9c9f285ea4c50" candidate_uuid: "0d321545cba84d2a934388a91a3e449b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e383e2c3234f788a8c3b9fb9b15230"
I20260811 06:26:19.498097 21868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e254a87888f14f8ca3c9c9f285ea4c50" candidate_uuid: "0d321545cba84d2a934388a91a3e449b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0d6dd16f3742488c6a1a703ac14c93"
I20260811 06:26:19.498272 21546 raft_consensus.cc:3037] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:19.498304 21868 raft_consensus.cc:3037] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:19.500497 21546 raft_consensus.cc:2454] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d321545cba84d2a934388a91a3e449b in term 1.
I20260811 06:26:19.500537 21868 raft_consensus.cc:2454] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d321545cba84d2a934388a91a3e449b in term 1.
I20260811 06:26:19.500878 21639 leader_election.cc:302] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d321545cba84d2a934388a91a3e449b, d3e383e2c3234f788a8c3b9fb9b15230; no voters:
I20260811 06:26:19.501124 21954 raft_consensus.cc:2781] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:19.501577 21954 raft_consensus.cc:686] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 1 LEADER]: Becoming Leader. State: Replica: 0d321545cba84d2a934388a91a3e449b, State: Running, Role: LEADER
I20260811 06:26:19.502090 21954 consensus_queue.cc:237] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } }
I20260811 06:26:19.506333 21378 catalog_manager.cc:5495] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d321545cba84d2a934388a91a3e449b (127.7.4.130). New cstate: current_term: 1 leader_uuid: "0d321545cba84d2a934388a91a3e449b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d321545cba84d2a934388a91a3e449b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 42943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 } health_report { overall_health: UNKNOWN } } }
I20260811 06:26:19.585249 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.579748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.7.4.130:42943)
0811 06:26:19.579941 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.579946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.580197 (+ 251us) server_negotiation.cc:407] Connection header received
0811 06:26:19.580437 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.580441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.580524 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.580647 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:26:19.581057 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.581815 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.583158 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.583622 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.583719 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.584191 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:26:19.584680 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:26:19.584801 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.584874 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.584937 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:26:19.593869 21868 raft_consensus.cc:1261] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 1 FOLLOWER]: Refusing update from remote peer 0d321545cba84d2a934388a91a3e449b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:19.593856 21546 raft_consensus.cc:1261] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 1 FOLLOWER]: Refusing update from remote peer 0d321545cba84d2a934388a91a3e449b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:19.594936 21965 consensus_queue.cc:1035] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e383e2c3234f788a8c3b9fb9b15230" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:19.595332 21954 consensus_queue.cc:1035] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0d6dd16f3742488c6a1a703ac14c93" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:19.619663 21378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:19.619961 21378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:19.623142 21378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ee36ace44674e2c92df9c2a6ae58027: Sending DeleteTablet for 3 replicas of tablet e254a87888f14f8ca3c9c9f285ea4c50
I20260811 06:26:19.624425 21690 tablet_service.cc:1514] Processing DeleteTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:37202
I20260811 06:26:19.624568 21843 tablet_service.cc:1514] Processing DeleteTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:51466
I20260811 06:26:19.625350 21525 tablet_service.cc:1514] Processing DeleteTablet for tablet e254a87888f14f8ca3c9c9f285ea4c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:53526
I20260811 06:26:19.625613 21982 tablet_replica.cc:330] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: stopping tablet replica
I20260811 06:26:19.626040 21982 raft_consensus.cc:2229] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:19.626538 21982 raft_consensus.cc:2258] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:19.628414 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21423
I20260811 06:26:19.631136 21982 ts_tablet_manager.cc:1905] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:19.639863 21983 tablet_replica.cc:330] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: stopping tablet replica
I20260811 06:26:19.640882 21983 raft_consensus.cc:2229] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:19.641531 21983 raft_consensus.cc:2258] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:19.642858 21981 tablet_replica.cc:330] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: stopping tablet replica
I20260811 06:26:19.643244 21981 raft_consensus.cc:2229] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:26:19.644104 21983 ts_tablet_manager.cc:1905] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:19.644594 21982 ts_tablet_manager.cc:1918] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:19.644768 21982 log.cc:1196] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/e254a87888f14f8ca3c9c9f285ea4c50
I20260811 06:26:19.645321 21982 ts_tablet_manager.cc:1939] T e254a87888f14f8ca3c9c9f285ea4c50 P ab0d6dd16f3742488c6a1a703ac14c93: Deleting consensus metadata
I20260811 06:26:19.645879 21981 raft_consensus.cc:2258] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:19.648708 21367 catalog_manager.cc:4841] TS ab0d6dd16f3742488c6a1a703ac14c93 (127.7.4.131:39431): tablet e254a87888f14f8ca3c9c9f285ea4c50 (table test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]) successfully deleted
I20260811 06:26:19.654057 21983 ts_tablet_manager.cc:1918] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:19.654775 21981 ts_tablet_manager.cc:1905] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:19.656669 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.629755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.7.4.129:34081)
0811 06:26:19.630089 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.630099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.640073 (+ 9974us) server_negotiation.cc:407] Connection header received
0811 06:26:19.640286 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.640293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.640400 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.640507 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:19.642423 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.643646 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.645596 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.646202 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.646357 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.649781 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:19.649792 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:19.649804 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:19.649835 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:19.653338 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.654153 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.654161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.654165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.654335 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.654727 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.654733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.654737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.655183 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:19.655573 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.655978 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.656224 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260811 06:26:19.659466 21983 log.cc:1196] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/e254a87888f14f8ca3c9c9f285ea4c50
I20260811 06:26:19.660319 21983 ts_tablet_manager.cc:1939] T e254a87888f14f8ca3c9c9f285ea4c50 P d3e383e2c3234f788a8c3b9fb9b15230: Deleting consensus metadata
I20260811 06:26:19.660416 21594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:19.662860 21981 ts_tablet_manager.cc:1918] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:19.663192 21981 log.cc:1196] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/e254a87888f14f8ca3c9c9f285ea4c50
I20260811 06:26:19.664410 21981 ts_tablet_manager.cc:1939] T e254a87888f14f8ca3c9c9f285ea4c50 P 0d321545cba84d2a934388a91a3e449b: Deleting consensus metadata
I20260811 06:26:19.666323 21364 catalog_manager.cc:4841] TS 0d321545cba84d2a934388a91a3e449b (127.7.4.130:42943): tablet e254a87888f14f8ca3c9c9f285ea4c50 (table test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]) successfully deleted
I20260811 06:26:19.767072 21366 catalog_manager.cc:4841] TS d3e383e2c3234f788a8c3b9fb9b15230 (127.7.4.129:34081): tablet e254a87888f14f8ca3c9c9f285ea4c50 (table test-table [id=ae53bfd298ba485f8a88f4d42ad9a619]) successfully deleted
I20260811 06:26:19.767316 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21423
I20260811 06:26:19.790060 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21604
I20260811 06:26:19.805768 21941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.790873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.7.4.130:42943)
0811 06:26:19.791102 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.791109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.791132 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:26:19.791279 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.791305 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.791421 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.791532 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:19.792807 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.794084 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.795434 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.796103 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.796238 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.798693 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:19.798712 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:19.798737 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:19.798790 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:19.802840 (+ 4050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.803592 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.803601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.803604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.803803 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.804176 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.804181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.804184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.804614 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:19.804964 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.805131 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.805373 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:26:19.806538 21763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:19.918969 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21604
I20260811 06:26:19.949720 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21772
I20260811 06:26:19.963475 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:19.950588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.7.4.131:39431)
0811 06:26:19.950792 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:19.950798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:19.950817 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:26:19.950922 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:19.950927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:19.950998 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:19.951108 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:19.952471 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.953293 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.954639 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:19.955040 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:19.955183 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:19.957790 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:19.957827 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:19.957841 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:19.957870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:19.960673 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.961341 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.961350 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.961359 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.961485 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:19.961867 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:19.961873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:19.961876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:19.962223 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:19.962524 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:19.962844 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 06:26:19.963056 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:26:19.964131 21915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:20.052443 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21772
I20260811 06:26:20.073216 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 21345
I20260811 06:26:20.074594 21415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:20.187594 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 21345
I20260811 06:26:20.214643 7186 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:26:20.560297 7186 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:26:20.560534 7186 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:26:20.906867 7186 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:26:20.907 UTC [20303] LOG: received smart shutdown request
2026-08-11 06:26:20.910 UTC [20303] LOG: background worker "logical replication launcher" (PID 20313) exited with exit code 1
2026-08-11 06:26:20.910 UTC [20308] LOG: shutting down
2026-08-11 06:26:21.073 UTC [20303] LOG: database system is shut down
2026-08-11T06:26:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52367 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:26:21 dist-test-slave-07c2 krb5kdc[22000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:26:21 dist-test-slave-07c2 krb5kdc[22000](info): set up 2 sockets
Aug 11 06:26:21 dist-test-slave-07c2 krb5kdc[22000](info): commencing operation
krb5kdc: starting...
W20260811 06:26:22.388198 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:26:22 dist-test-slave-07c2 krb5kdc[22000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:26:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:26:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:26:22.651878 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37351
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:22.983206 22015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:22.983554 22015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:22.983726 22015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:22.990839 22015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:22.990935 22015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:22.991037 22015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:22.991130 22015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:22.991214 22015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:23.000772 22015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:37351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:23.001958 22015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:23.003302 22015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:23.015134 22022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:23.015089 22025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:23.015163 22023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:23.016198 22015 server_base.cc:1034] running on GCE node
I20260811 06:26:23.016892 22015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:23.018080 22015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:23.019356 22015 hybrid_clock.cc:648] HybridClock initialized: now 1786429583019275 us; error 50 us; skew 500 ppm
I20260811 06:26:23.019699 22015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:23.027410 22015 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:23.028782 22015 webserver.cc:460] Webserver started at http://127.7.4.190:39847/ using document root <none> and password file <none>
I20260811 06:26:23.029405 22015 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:23.029503 22015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:23.029764 22015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:23.032665 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "9181bfa36a614a12b1a73a8baf4d4db0"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "9fce44b619a83f06b0dabfb8727d5970"
server_key_iv: "3a431d9f06d8037f5e4aa05ce27ad156"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.033438 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "9181bfa36a614a12b1a73a8baf4d4db0"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "9fce44b619a83f06b0dabfb8727d5970"
server_key_iv: "3a431d9f06d8037f5e4aa05ce27ad156"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.038389 22015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:26:23.041579 22032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:23.042809 22015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:26:23.042935 22015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "9181bfa36a614a12b1a73a8baf4d4db0"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "9fce44b619a83f06b0dabfb8727d5970"
server_key_iv: "3a431d9f06d8037f5e4aa05ce27ad156"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.043102 22015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:23.057744 22015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:23.061390 22015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:23.061611 22015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:23.079759 22015 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:37351
I20260811 06:26:23.079802 22089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:37351 every 8 connection(s)
I20260811 06:26:23.081035 22015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:23.084515 22090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:23.084836 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22015
I20260811 06:26:23.085044 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:26:23.085716 7186 external_mini_cluster.cc:1428] Setting key b5e46e9c3382152c9af095925857735a
I20260811 06:26:23.090870 22090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.4.189@KRBTEST.COM; defaulting to no policy
I20260811 06:26:23.112416 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.093856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37351 (local address 127.0.0.1:60928)
0811 06:26:23.095029 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.095068 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:23.096744 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:23.097520 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:23.097526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:23.098205 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:23.098522 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.098538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.100104 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.100111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.101156 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.101168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.102050 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.102057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.102231 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.102895 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:23.102916 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:23.105313 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:23.108318 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.108330 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.108340 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.108746 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.109145 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.109151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.109155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.109310 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.110013 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:23.110020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:23.110246 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 06:26:23.110480 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 06:26:23.111054 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":165,"spinlock_wait_cycles":16640,"thread_start_us":785,"threads_started":1}
I20260811 06:26:23.112396 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.093855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.7.4.190:37351)
0811 06:26:23.094724 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:23.094742 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:23.095178 (+ 436us) server_negotiation.cc:407] Connection header received
0811 06:26:23.097092 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:23.097098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:23.097310 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:23.097505 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:23.098712 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.099957 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.101296 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.101825 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.102027 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.105518 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:23.105525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:23.105535 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:23.105561 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:23.108192 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.108886 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.108892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.108894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.109039 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.109421 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.109430 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.109432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.109877 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:23.110082 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:23.110597 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 06:26:23.110953 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":646,"thread_start_us":444,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.189@KRBTEST.COM" created.
I20260811 06:26:23.115129 22090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:26:23.116127 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.115739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53602)
0811 06:26:23.115903 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.116024 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 06:26:23.116341 22035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
I20260811 06:26:23.117883 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.117552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53612)
0811 06:26:23.117686 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.117763 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 06:26:23.118101 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:44361: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
I20260811 06:26:23.144217 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:44361
--webserver_interface=127.7.4.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:23.167560 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } attempt: 1
I20260811 06:26:23.168808 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.168365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53616)
0811 06:26:23.168571 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.168678 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:26:23.169799 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.169437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53632)
0811 06:26:23.169591 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.169677 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:26:23.170006 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:44361: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
I20260811 06:26:23.238221 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } attempt: 2
I20260811 06:26:23.239437 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.238869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53636)
0811 06:26:23.239139 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.239233 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:26:23.240459 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.240088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53642)
0811 06:26:23.240265 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.240328 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:26:23.240651 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:44361: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
I20260811 06:26:23.331902 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } attempt: 3
I20260811 06:26:23.333069 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.332617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53652)
0811 06:26:23.332828 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.332926 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:26:23.334136 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.333765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53668)
0811 06:26:23.333909 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.333989 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:26:23.334314 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:44361: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
I20260811 06:26:23.477574 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } attempt: 4
I20260811 06:26:23.478819 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.478300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53678)
0811 06:26:23.478599 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.478688 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260811 06:26:23.479982 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.479625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53682)
0811 06:26:23.479763 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.479850 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:26:23.480244 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:44361: Network error: Client connection negotiation failed: client connection to 127.7.4.189:44361: connect: Connection refused (error 111)
W20260811 06:26:23.485311 22098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:23.485615 22098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:23.485775 22098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:23.493656 22098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:23.493777 22098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:23.493863 22098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:23.493970 22098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:23.494032 22098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:23.504012 22098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.4.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:23.505386 22098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:23.506738 22098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:23.518710 22107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:23.518782 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:23.519672 22106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:23.520026 22098 server_base.cc:1034] running on GCE node
I20260811 06:26:23.520761 22098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:23.522104 22098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:23.523365 22098 hybrid_clock.cc:648] HybridClock initialized: now 1786429583523297 us; error 57 us; skew 500 ppm
I20260811 06:26:23.523891 22098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:23.527282 22098 init.cc:377] Logged in from keytab as kudu/127.7.4.189@KRBTEST.COM (short username kudu)
I20260811 06:26:23.529060 22098 webserver.cc:460] Webserver started at http://127.7.4.189:35711/ using document root <none> and password file <none>
I20260811 06:26:23.529811 22098 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:23.529959 22098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:23.530273 22098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:23.533404 22098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data/instance:
uuid: "842a9bbe55a8432983b4bd09b06416ff"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "6f35f889bcec25413bf59d743f53c0ce"
server_key_iv: "bc1272be48388af726ef19058b979f0b"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.534271 22098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal/instance:
uuid: "842a9bbe55a8432983b4bd09b06416ff"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "6f35f889bcec25413bf59d743f53c0ce"
server_key_iv: "bc1272be48388af726ef19058b979f0b"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.539904 22098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:26:23.543488 22116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:23.544768 22098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:26:23.544955 22098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "842a9bbe55a8432983b4bd09b06416ff"
format_stamp: "Formatted at 2026-08-11 06:26:23 on dist-test-slave-07c2"
server_key: "6f35f889bcec25413bf59d743f53c0ce"
server_key_iv: "bc1272be48388af726ef19058b979f0b"
server_key_version: "encryptionkey@0"
I20260811 06:26:23.545164 22098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:23.580058 22098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:23.584595 22098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:23.584896 22098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:23.603899 22098 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.189:44361
I20260811 06:26:23.603941 22171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.189:44361 every 8 connection(s)
I20260811 06:26:23.605343 22098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
I20260811 06:26:23.609091 22172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:23.610443 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22098
I20260811 06:26:23.610653 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-1/wal/instance
I20260811 06:26:23.611238 7186 external_mini_cluster.cc:1428] Setting key 451fd2a396c60f6b11dfb75e1579eae4
I20260811 06:26:23.618933 22172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.4.188@KRBTEST.COM; defaulting to no policy
I20260811 06:26:23.643541 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.622417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.7.4.190:37351)
0811 06:26:23.622956 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:23.622962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:23.624031 (+ 1069us) server_negotiation.cc:407] Connection header received
0811 06:26:23.626883 (+ 2852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:23.626888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:23.626970 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:23.627058 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:23.629250 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.630034 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.632613 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.633027 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.633124 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.637389 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:23.637393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:23.637397 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:23.637435 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:23.640351 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.641110 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.641114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.641117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.641236 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.641666 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.641671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.641678 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.641969 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:23.642123 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:23.642881 (+ 758us) server_negotiation.cc:299] Negotiation successful
0811 06:26:23.643108 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":257,"threads_started":1}
I20260811 06:26:23.644586 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.622975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37351 (local address 127.0.0.1:60936)
0811 06:26:23.623851 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.623898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:23.626551 (+ 2653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:23.627351 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:23.627365 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:23.628185 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:23.629038 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.629087 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.630205 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.630213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.632470 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.632489 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.633266 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.633276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.633605 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.634365 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:23.634399 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:23.637137 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:23.640496 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.640505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.640522 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.640973 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.641351 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.641356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.641359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.641537 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.642101 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:23.642108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:23.642473 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 06:26:23.642781 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 06:26:23.643346 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":403,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.188@KRBTEST.COM" created.
I20260811 06:26:23.646531 22172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:26:23.663987 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.647438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53694)
0811 06:26:23.647630 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.647652 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:23.647765 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:23.649143 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:23.649148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:23.649669 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:23.649908 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.649922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.651862 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.651870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.653371 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.653389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.654503 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.654512 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.654767 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.655455 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:23.655485 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:23.658163 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:23.661416 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.661423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.661427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.661788 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.662154 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.662160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.662163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.662311 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.662909 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:23.662917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:23.663063 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 06:26:23.663199 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:26:23.663563 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:26:23.664069 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.647582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.7.4.189:44361)
0811 06:26:23.648288 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:23.648298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:23.648331 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:26:23.648387 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:23.648435 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:23.648689 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:23.648865 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:23.650080 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.651690 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.653506 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.654339 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.654471 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.658387 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:23.658399 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:23.658412 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:23.658447 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:23.661286 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.661917 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.661926 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.661929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.662050 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.662419 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.662426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.662429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.662790 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:23.662999 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:23.663392 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 06:26:23.663676 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":519,"thread_start_us":3800,"threads_started":1}
I20260811 06:26:23.666397 22172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:26:23.667573 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.667074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51926)
0811 06:26:23.667247 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.667432 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:26:23.667809 22118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.669378 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.669047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51928)
0811 06:26:23.669197 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.669274 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:26:23.669605 22172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
I20260811 06:26:23.679931 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:36271
--webserver_interface=127.7.4.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:23.718880 22172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 1
I20260811 06:26:23.720127 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.719553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51942)
0811 06:26:23.719778 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.719908 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:26:23.721253 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.720861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51944)
0811 06:26:23.721030 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.721117 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:26:23.721477 22172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.779518 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } attempt: 5
Aug 11 06:26:23 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:26:23.789706 22172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 2
I20260811 06:26:23.790782 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.790251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51960)
0811 06:26:23.790435 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.790557 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:26:23.791855 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.791508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51964)
0811 06:26:23.791657 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.791736 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:26:23.792126 22172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.794818 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.780317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53702)
0811 06:26:23.780551 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.780574 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:23.780742 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:23.781193 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:23.781218 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:23.781854 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:23.782152 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.782170 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.783753 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.783759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.784655 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:23.784673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.785622 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.785626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:23.785787 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.786412 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:23.786429 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:23.788486 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:23.792567 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.792571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.792573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.792934 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.793322 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:23.793325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:23.793327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:23.793466 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:23.794127 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:23.794130 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:23.794249 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 06:26:23.794388 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:26:23.794585 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 06:26:23.795449 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.780368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.7.4.189:44361)
0811 06:26:23.780568 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:23.780576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:23.780663 (+ 87us) server_negotiation.cc:407] Connection header received
0811 06:26:23.780902 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:23.780906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:23.780985 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:23.781097 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:23.782374 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.783598 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.784809 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:23.785464 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:23.785616 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:23.788696 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:23.788703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:23.788708 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:23.788749 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:23.792443 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.793053 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.793058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.793062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.793214 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:23.793582 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:23.793590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:23.793595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:23.794013 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:23.794185 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:23.794561 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 06:26:23.794830 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:26:23.796450 22090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:26:23.797382 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.797015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51970)
0811 06:26:23.797175 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.797247 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:26:23.798199 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.797900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:51986)
0811 06:26:23.798040 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.798114 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 06:26:23.798413 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.849606 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 1
I20260811 06:26:23.850878 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.850278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52002)
0811 06:26:23.850515 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.850649 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:26:23.851997 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.851590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52016)
0811 06:26:23.851763 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.851859 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:26:23.852248 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.883381 22172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 3
I20260811 06:26:23.884490 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.883995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52028)
0811 06:26:23.884228 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.884325 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:26:23.885557 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.885139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52034)
0811 06:26:23.885376 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.885448 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 06:26:23.885802 22172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:23.920620 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 2
I20260811 06:26:23.921845 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.921237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52048)
0811 06:26:23.921501 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.921629 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:26:23.922921 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:23.922539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52062)
0811 06:26:23.922707 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:23.922795 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:26:23.923209 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
W20260811 06:26:24.022009 22181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:24.022300 22181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:24.022435 22181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 06:26:24.029021 22172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 4
I20260811 06:26:24.029453 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 3
W20260811 06:26:24.029583 22181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:24.029707 22181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:24.029812 22181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:24.029917 22181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:24.029994 22181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:24.030411 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.029743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52078)
0811 06:26:24.030119 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.030227 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 06:26:24.030426 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.030023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52088)
0811 06:26:24.030183 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.030269 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 06:26:24.031512 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.031115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52096)
0811 06:26:24.031274 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.031382 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:26:24.031545 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.031128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52112)
0811 06:26:24.031375 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.031445 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260811 06:26:24.031776 22172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
W20260811 06:26:24.031778 22090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:36271: Network error: Client connection negotiation failed: client connection to 127.7.4.188:36271: connect: Connection refused (error 111)
I20260811 06:26:24.040977 22181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:37351,127.7.4.189:44361,127.7.4.188:36271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:36271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.4.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:24.042250 22181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:24.043712 22181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:24.055855 22186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:24.055809 22190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:24.055894 22188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:24.056787 22181 server_base.cc:1034] running on GCE node
I20260811 06:26:24.057624 22181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:24.058952 22181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:24.060171 22181 hybrid_clock.cc:648] HybridClock initialized: now 1786429584060123 us; error 56 us; skew 500 ppm
I20260811 06:26:24.060523 22181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:24.063719 22181 init.cc:377] Logged in from keytab as kudu/127.7.4.188@KRBTEST.COM (short username kudu)
I20260811 06:26:24.065117 22181 webserver.cc:460] Webserver started at http://127.7.4.188:45283/ using document root <none> and password file <none>
I20260811 06:26:24.065752 22181 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:24.065907 22181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:24.066174 22181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:24.069077 22181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data/instance:
uuid: "56799c8db1b14bc891de8935ce7d0ef1"
format_stamp: "Formatted at 2026-08-11 06:26:24 on dist-test-slave-07c2"
server_key: "0167506deefb2838fd6dd2ac179003db"
server_key_iv: "31329de0f2be31117b155fbe2dddaf34"
server_key_version: "encryptionkey@0"
I20260811 06:26:24.069823 22181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal/instance:
uuid: "56799c8db1b14bc891de8935ce7d0ef1"
format_stamp: "Formatted at 2026-08-11 06:26:24 on dist-test-slave-07c2"
server_key: "0167506deefb2838fd6dd2ac179003db"
server_key_iv: "31329de0f2be31117b155fbe2dddaf34"
server_key_version: "encryptionkey@0"
I20260811 06:26:24.074923 22181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:26:24.078305 22197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:24.079627 22181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:24.079806 22181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "56799c8db1b14bc891de8935ce7d0ef1"
format_stamp: "Formatted at 2026-08-11 06:26:24 on dist-test-slave-07c2"
server_key: "0167506deefb2838fd6dd2ac179003db"
server_key_iv: "31329de0f2be31117b155fbe2dddaf34"
server_key_version: "encryptionkey@0"
I20260811 06:26:24.079996 22181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:24.094805 22181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:24.099138 22181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:24.099465 22181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:24.117780 22181 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.188:36271
I20260811 06:26:24.117820 22252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.188:36271 every 8 connection(s)
I20260811 06:26:24.119129 22181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
I20260811 06:26:24.122653 22253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:24.127032 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22181
I20260811 06:26:24.127230 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429373903770-7186-0/minicluster-data/master-2/wal/instance
I20260811 06:26:24.127704 7186 external_mini_cluster.cc:1428] Setting key 2b4d7a47c4d10212d747f8863dba29f1
I20260811 06:26:24.129508 22253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:26:24.148283 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.132074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.7.4.190:37351)
0811 06:26:24.132239 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.132244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.132983 (+ 739us) server_negotiation.cc:407] Connection header received
0811 06:26:24.133332 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.133340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.133429 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.133533 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.134806 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.135552 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.137017 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.137454 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.137587 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.141429 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.141435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.141441 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.141484 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.144644 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.145521 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.145524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.145526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.145644 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.146152 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.146161 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.146163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.146454 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.146612 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.147660 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.147901 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:26:24.149366 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.132105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:37351 (local address 127.0.0.1:60942)
0811 06:26:24.132844 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.132876 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:24.133105 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:24.133596 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:24.133602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:24.134242 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:24.134615 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.134631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.135714 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.135718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.136875 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.136887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.137708 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.137716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.137909 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.138631 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:24.138672 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:24.141208 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:24.144779 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.144796 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.144809 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.145400 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.145756 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.145760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.145764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.146018 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.146626 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:24.146634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:24.147113 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 06:26:24.147513 (+ 400us) client_negotiation.cc:240] Negotiation successful
0811 06:26:24.147938 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"spinlock_wait_cycles":12672,"thread_start_us":420,"threads_started":1}
I20260811 06:26:24.149925 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.129464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.7.4.188:36271)
0811 06:26:24.130413 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.130427 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.130472 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:26:24.132703 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.132726 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.133148 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.133367 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.134860 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.136100 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.137410 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.137955 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.138096 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.142789 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.142797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.142808 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.142836 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.146017 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.146797 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.146804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.146806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.146930 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.147415 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.147419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.147421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.147926 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.148292 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.149333 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.149585 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":468,"threads_started":1}
I20260811 06:26:24.150969 22253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:24.165503 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.151803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:44361 (local address 127.0.0.1:53708)
0811 06:26:24.151988 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.152007 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:24.152097 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:24.152457 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:24.152462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:24.152901 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:24.153102 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.153115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.154504 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.154508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.155970 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.155985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.156912 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.156916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.157051 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.157603 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:24.157620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:24.159824 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:24.163069 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.163075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.163079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.163523 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.163903 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.163908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.163911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.164062 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.164559 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:24.164565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:24.164713 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 06:26:24.164849 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:26:24.165126 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:26:24.165550 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.151815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.7.4.189:44361)
0811 06:26:24.152013 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.152021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.152057 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:26:24.152231 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.152236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.152308 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.152422 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.153257 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.154346 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.156103 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.156778 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.156905 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.160013 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.160018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.160021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.160054 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.162934 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.163636 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.163643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.163647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.163793 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.164181 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.164185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.164187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.164441 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.164603 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.164974 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.165188 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:26:24.166800 22253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:26:24.167678 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.152774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.7.4.190:37351)
0811 06:26:24.152918 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.152921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.152959 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:26:24.153127 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.153130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.153187 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.153267 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.154712 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.155847 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.157468 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.157902 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.158029 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.162107 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.162111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.162114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.162144 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.164754 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.165450 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.165457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.165460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.165659 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.166093 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.166098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.166101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.166435 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.166713 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.167025 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.167256 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:26:24.180653 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.167788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.7.4.188:36271)
0811 06:26:24.167965 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.167971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.167988 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:26:24.168033 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.168038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.168107 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.168203 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.169076 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.170014 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.171513 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.172165 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.172282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.175356 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.175361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.175364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.175392 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.177978 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.178612 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.178616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.178618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.178740 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.179167 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.179173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.179178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.179541 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.179653 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.180107 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.180376 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":65}
I20260811 06:26:24.181226 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.167579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52136)
0811 06:26:24.167735 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.167768 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:24.167877 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:24.168269 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:24.168273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:24.168770 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:24.168941 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.168952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.170149 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.170154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.171339 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.171354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.172296 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.172301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.172463 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.173012 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:24.173028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:24.175143 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:24.178105 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.178110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.178114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.178502 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.178857 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.178862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.178868 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.179028 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.179676 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:24.179682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:24.179839 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 06:26:24.180015 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:26:24.180585 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":186}
I20260811 06:26:24.184937 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.169488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.7.4.189:44361)
0811 06:26:24.169631 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.169635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.169653 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:26:24.169792 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.169795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.169844 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.169915 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.171364 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.172391 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.173545 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.174006 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.174139 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.178542 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.178550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.178556 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.178599 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.182002 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.182877 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.182883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.182889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.183050 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.183513 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.183517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.183520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.183893 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.184191 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.184369 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.184589 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:26:24.187520 22253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: Bootstrap starting.
I20260811 06:26:24.189625 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:26:24.190973 22253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:24.192054 22253 log.cc:824] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:24.195333 22253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: No bootstrap required, opened a new log
I20260811 06:26:24.209203 22253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.209667 22253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:24.209818 22253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56799c8db1b14bc891de8935ce7d0ef1, State: Initialized, Role: FOLLOWER
I20260811 06:26:24.210271 22090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 4
I20260811 06:26:24.210569 22253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.218281 22267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.218698 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.194600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.7.4.190:37351)
0811 06:26:24.194807 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.194813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.195362 (+ 549us) server_negotiation.cc:407] Connection header received
0811 06:26:24.195633 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.195639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.195714 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.195812 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.197387 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.198174 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.199943 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.200641 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.200753 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.206042 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.206049 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.206054 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.206113 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.210136 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.215866 (+ 5730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.215873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.215876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.216033 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.216608 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.216614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.216617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.217059 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.217359 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.218037 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.218280 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:26:24.219251 22267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.221092 22253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:24.223636 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.194971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.7.4.189:44361)
0811 06:26:24.195145 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.195151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.199903 (+ 4752us) server_negotiation.cc:407] Connection header received
0811 06:26:24.200143 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.200148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.200220 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.200289 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.203308 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.204636 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.206878 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.207914 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.208028 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.212671 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.212681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.212688 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.212738 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.216840 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.218562 (+ 1722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.218570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.218574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.218734 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.219997 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.220005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.220009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.220411 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.220643 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.222957 (+ 2314us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.223221 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:26:24.227047 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.199990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.7.4.188:36271)
0811 06:26:24.200187 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.200193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.207033 (+ 6840us) server_negotiation.cc:407] Connection header received
0811 06:26:24.207462 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.207467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.207556 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.207620 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.209499 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.210603 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.213156 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.214175 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.214308 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.219275 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.219301 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.219310 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.219347 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.223022 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.224700 (+ 1678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.224707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.224711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.224886 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.225363 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.225368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.225371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.225795 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.226034 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.226659 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.226807 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:26:24.234617 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.210822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52160)
0811 06:26:24.211144 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.211162 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:24.211261 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:24.216436 (+ 5175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:24.216442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:24.217009 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:24.217233 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.217247 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.220767 (+ 3520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.220772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.222177 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.222195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.223580 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.223589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.223833 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.224451 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:24.224480 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:24.228025 (+ 3545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:24.231716 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.231721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.231724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.232229 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.232737 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.232743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.232746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.232932 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.233607 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:24.233611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:24.233726 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 06:26:24.233892 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 06:26:24.234198 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 06:26:24.235591 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.211490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.7.4.188:36271)
0811 06:26:24.215248 (+ 3758us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.215255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.215280 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:26:24.215351 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.215357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.215445 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.217449 (+ 2004us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.217745 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.219069 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.222390 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.223375 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.224174 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.228234 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.228245 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.228249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.228287 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.231549 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.232415 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.232422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.232428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.232587 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.233072 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.233079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.233082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.233467 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.233606 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.234893 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.235120 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3609,"thread_start_us":3372,"threads_started":1}
W20260811 06:26:24.238749 22280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:26:24.238879 22280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:26:24.234133 22181 master_runner.cc:186] Error getting master registration for 127.7.4.190:37351: OK, instance_id { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" instance_seqno: 1786429583074685 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:26:24.244278 22181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:26:24.244686 22090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: Bootstrap starting.
I20260811 06:26:24.247925 22090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:24.249037 22090 log.cc:824] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:24.257380 22090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: No bootstrap required, opened a new log
I20260811 06:26:24.264446 22090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.264755 22090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:24.264873 22090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9181bfa36a614a12b1a73a8baf4d4db0, State: Initialized, Role: FOLLOWER
I20260811 06:26:24.265388 22090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.266266 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.266677 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.267270 22090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:26:24.282135 22295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:26:24.282364 22295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:26:24.275626 22015 master_runner.cc:186] Error getting master registration for 127.7.4.189:44361: OK, instance_id { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" instance_seqno: 1786429583598380 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:26:24.286012 22015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:26:24.314397 22038 tablet.cc:2369] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:24.331003 22172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } attempt: 5
Aug 11 06:26:24 dist-test-slave-07c2 krb5kdc[22000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:26:24.348913 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.331655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:36271 (local address 127.0.0.1:52170)
0811 06:26:24.332047 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:24.332079 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:24.332193 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:24.333085 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:24.333090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:24.333631 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:24.333838 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.333850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.335376 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.335382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.336829 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:24.336844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.337883 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.337890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:24.338100 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.338806 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:24.338831 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:24.341742 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:24.345899 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.345904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.345909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.346439 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.346859 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:24.346864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:24.346868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:24.347029 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:24.347781 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:24.347786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:24.347961 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 06:26:24.348131 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 06:26:24.348405 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 06:26:24.349337 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:24.332417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52170 (local address 127.7.4.188:36271)
0811 06:26:24.332601 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:24.332607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:24.332624 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:26:24.332682 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:24.332688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:24.332783 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:24.332883 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:24.334014 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.335222 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.336986 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:24.337720 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:24.337855 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:24.341962 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:24.341970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:24.341976 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:24.342014 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:24.345766 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.346539 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.346545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.346549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.346687 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:24.347165 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:24.347170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:24.347176 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:24.347646 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:24.347787 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:24.348297 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 06:26:24.348541 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 06:26:24.351670 22207 tablet.cc:2369] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:24.355408 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: Bootstrap starting.
I20260811 06:26:24.359568 22172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:24.360900 22172 log.cc:824] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:24.364168 22172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: No bootstrap required, opened a new log
I20260811 06:26:24.375393 22172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.375790 22172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:24.375918 22172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842a9bbe55a8432983b4bd09b06416ff, State: Initialized, Role: FOLLOWER
I20260811 06:26:24.376528 22172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.377516 22298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.378110 22298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.378831 22172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:24.394680 22098 master_runner.cc:263] No leader master found from master 842a9bbe55a8432983b4bd09b06416ff
I20260811 06:26:24.394821 22098 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:26:24.397920 22313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:26:24.398051 22313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:26:24.480006 22298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:24.480402 22298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.481984 22298 leader_election.cc:288] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9181bfa36a614a12b1a73a8baf4d4db0 (127.7.4.190:37351), 56799c8db1b14bc891de8935ce7d0ef1 (127.7.4.188:36271)
I20260811 06:26:24.483680 22063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "842a9bbe55a8432983b4bd09b06416ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" is_pre_election: true
I20260811 06:26:24.484328 22063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842a9bbe55a8432983b4bd09b06416ff in term 0.
I20260811 06:26:24.485178 22228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "842a9bbe55a8432983b4bd09b06416ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56799c8db1b14bc891de8935ce7d0ef1" is_pre_election: true
I20260811 06:26:24.485800 22228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842a9bbe55a8432983b4bd09b06416ff in term 0.
I20260811 06:26:24.486507 22118 leader_election.cc:302] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56799c8db1b14bc891de8935ce7d0ef1, 842a9bbe55a8432983b4bd09b06416ff; no voters:
I20260811 06:26:24.487318 22298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:26:24.487474 22298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:26:24.487574 22298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:24.491214 22298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.492463 22063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "842a9bbe55a8432983b4bd09b06416ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9181bfa36a614a12b1a73a8baf4d4db0"
I20260811 06:26:24.492681 22063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:24.499711 22063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842a9bbe55a8432983b4bd09b06416ff in term 1.
I20260811 06:26:24.500216 22118 leader_election.cc:302] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842a9bbe55a8432983b4bd09b06416ff, 9181bfa36a614a12b1a73a8baf4d4db0; no voters:
I20260811 06:26:24.503140 22298 leader_election.cc:288] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [CANDIDATE]: Term 1 election: Requested vote from peers 9181bfa36a614a12b1a73a8baf4d4db0 (127.7.4.190:37351), 56799c8db1b14bc891de8935ce7d0ef1 (127.7.4.188:36271)
I20260811 06:26:24.503623 22298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:24.503980 22298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [term 1 LEADER]: Becoming Leader. State: Replica: 842a9bbe55a8432983b4bd09b06416ff, State: Running, Role: LEADER
I20260811 06:26:24.504655 22298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } }
I20260811 06:26:24.514956 22228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "842a9bbe55a8432983b4bd09b06416ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56799c8db1b14bc891de8935ce7d0ef1"
I20260811 06:26:24.515218 22228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:24.518828 22228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842a9bbe55a8432983b4bd09b06416ff in term 1.
I20260811 06:26:24.523566 22315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842a9bbe55a8432983b4bd09b06416ff. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.524109 22315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:24.533401 22317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:24.535960 22317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:24.558610 22228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [term 1 FOLLOWER]: Refusing update from remote peer 842a9bbe55a8432983b4bd09b06416ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:24.559841 22315 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:24.568238 22063 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [term 1 FOLLOWER]: Refusing update from remote peer 842a9bbe55a8432983b4bd09b06416ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:24.569442 22315 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:26:24.587538 22122 tablet.cc:2369] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:24.606575 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842a9bbe55a8432983b4bd09b06416ff. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.606928 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.615432 22318 mvcc.cc:205] Tried to move back new op lower bound from 7317215578339463168 to 7317215578142281728. Current Snapshot: MvccSnapshot[applied={T|T < 7317215578339463168}]
I20260811 06:26:24.624441 22320 mvcc.cc:205] Tried to move back new op lower bound from 7317215578339463168 to 7317215578142281728. Current Snapshot: MvccSnapshot[applied={T|T < 7317215578339463168}]
I20260811 06:26:24.624168 22298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.625342 22298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:24.632483 22284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.632743 22284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.633409 22317 catalog_manager.cc:1347] Generated new cluster ID: 44405b9695a0432fb46655b842d03916
I20260811 06:26:24.633522 22317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:24.653929 22267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842a9bbe55a8432983b4bd09b06416ff. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.654266 22267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.657162 22298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.657486 22298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:24.667827 22317 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:26:24.669895 22317 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:24.675678 22267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "842a9bbe55a8432983b4bd09b06416ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9181bfa36a614a12b1a73a8baf4d4db0" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 37351 } } peers { permanent_uuid: "842a9bbe55a8432983b4bd09b06416ff" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 44361 } } peers { permanent_uuid: "56799c8db1b14bc891de8935ce7d0ef1" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 36271 } } }
I20260811 06:26:24.675944 22267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:26:24.700815 22317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 842a9bbe55a8432983b4bd09b06416ff: Generated new TSK 0
I20260811 06:26:24.702014 22317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:26:25.240911 22280 catalog_manager.cc:1259] Loaded cluster ID: 44405b9695a0432fb46655b842d03916
I20260811 06:26:25.241036 22280 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: loading cluster ID for follower catalog manager: success
I20260811 06:26:25.243582 22280 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: acquiring CA information for follower catalog manager: success
I20260811 06:26:25.245229 22280 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56799c8db1b14bc891de8935ce7d0ef1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:26:25.284302 22295 catalog_manager.cc:1259] Loaded cluster ID: 44405b9695a0432fb46655b842d03916
I20260811 06:26:25.284389 22295 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: loading cluster ID for follower catalog manager: success
I20260811 06:26:25.286602 22295 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: acquiring CA information for follower catalog manager: success
I20260811 06:26:25.288298 22295 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9181bfa36a614a12b1a73a8baf4d4db0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:26:25.310684 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22015
I20260811 06:26:25.311770 22084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:25.420223 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22015
I20260811 06:26:25.446991 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22098
I20260811 06:26:25.448086 22167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:25.557096 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22098
I20260811 06:26:25.586311 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22181
I20260811 06:26:25.587638 22248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:25.696026 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22181
2026-08-11T06:26:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4493 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:26:25 dist-test-slave-07c2 krb5kdc[22340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:26:25 dist-test-slave-07c2 krb5kdc[22340](info): set up 2 sockets
Aug 11 06:26:25 dist-test-slave-07c2 krb5kdc[22340](info): commencing operation
krb5kdc: starting...
W20260811 06:26:26.891362 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:26:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:26:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:26:27.182811 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44015
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:44015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:27.513226 22356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:27.513494 22356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:27.513607 22356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:27.520663 22356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:27.520767 22356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:27.520885 22356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:27.520980 22356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:27.521056 22356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:27.530821 22356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:44015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:27.532186 22356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:27.533555 22356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:27.544929 22363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:27.544984 22365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:27.545006 22362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:27.545892 22356 server_base.cc:1034] running on GCE node
I20260811 06:26:27.546765 22356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:27.548153 22356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:27.549413 22356 hybrid_clock.cc:648] HybridClock initialized: now 1786429587549366 us; error 49 us; skew 500 ppm
I20260811 06:26:27.549783 22356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429587, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:27.552999 22356 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:27.554524 22356 webserver.cc:460] Webserver started at http://127.7.4.190:43099/ using document root <none> and password file <none>
I20260811 06:26:27.555184 22356 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:27.555347 22356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:27.555667 22356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:27.558486 22356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "6758666981c04a08809baf6d7695d62c"
format_stamp: "Formatted at 2026-08-11 06:26:27 on dist-test-slave-07c2"
server_key: "947756db885fe22bae63a194a0c363be"
server_key_iv: "d8794e57f9d5d4d38f0f413bdabe001d"
server_key_version: "encryptionkey@0"
I20260811 06:26:27.559221 22356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "6758666981c04a08809baf6d7695d62c"
format_stamp: "Formatted at 2026-08-11 06:26:27 on dist-test-slave-07c2"
server_key: "947756db885fe22bae63a194a0c363be"
server_key_iv: "d8794e57f9d5d4d38f0f413bdabe001d"
server_key_version: "encryptionkey@0"
I20260811 06:26:27.564303 22356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:26:27.567622 22374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:27.568866 22356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:26:27.569046 22356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "6758666981c04a08809baf6d7695d62c"
format_stamp: "Formatted at 2026-08-11 06:26:27 on dist-test-slave-07c2"
server_key: "947756db885fe22bae63a194a0c363be"
server_key_iv: "d8794e57f9d5d4d38f0f413bdabe001d"
server_key_version: "encryptionkey@0"
I20260811 06:26:27.569238 22356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:27.607837 22356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:27.612166 22356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:27.612468 22356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:27.630946 22356 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:44015
I20260811 06:26:27.630992 22432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:44015 every 8 connection(s)
I20260811 06:26:27.632350 22356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:27.635923 22433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:27.640326 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22356
I20260811 06:26:27.640580 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:26:27.641439 7186 external_mini_cluster.cc:1428] Setting key be5d7cf1a275c80184498bbe8ae94994
I20260811 06:26:27.647445 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c: Bootstrap starting.
I20260811 06:26:27.650363 22433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:27.651373 22433 log.cc:824] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:27.653939 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c: No bootstrap required, opened a new log
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:27.661590 22433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } }
I20260811 06:26:27.661950 22433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:27.662076 22433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6758666981c04a08809baf6d7695d62c, State: Initialized, Role: FOLLOWER
I20260811 06:26:27.662720 22433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } }
I20260811 06:26:27.663046 22433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:27.663208 22433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:27.663420 22433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:27.664593 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:27.643266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.7.4.190:44015)
0811 06:26:27.644083 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:27.644098 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:27.644137 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:26:27.646082 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:27.646103 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:27.646427 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:27.646813 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:27.648417 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:27.649701 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:27.651128 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:27.651878 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:27.652090 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:27.656534 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:27.656541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:27.656561 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:27.656602 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:27.659801 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:27.660607 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:27.660617 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:27.660622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:27.660892 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:27.661328 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:27.661333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:27.661336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:27.661896 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:27.662205 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:27.662748 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 06:26:27.663213 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":362,"threads_started":1}
I20260811 06:26:27.666115 22433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } }
I20260811 06:26:27.666620 22433 leader_election.cc:302] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6758666981c04a08809baf6d7695d62c; no voters:
I20260811 06:26:27.667274 22433 leader_election.cc:288] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:26:27.667397 22438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:27.668668 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:26:27.669275 22433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:27.673743 22438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [term 1 LEADER]: Becoming Leader. State: Replica: 6758666981c04a08809baf6d7695d62c, State: Running, Role: LEADER
I20260811 06:26:27.674273 22438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } }
I20260811 06:26:27.676990 22440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6758666981c04a08809baf6d7695d62c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } } }
I20260811 06:26:27.677510 22440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:27.676966 22441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6758666981c04a08809baf6d7695d62c. Latest consensus state: current_term: 1 leader_uuid: "6758666981c04a08809baf6d7695d62c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6758666981c04a08809baf6d7695d62c" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44015 } } }
I20260811 06:26:27.677781 22441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:27.679323 22454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:27.683535 22454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:27.690614 22454 catalog_manager.cc:1347] Generated new cluster ID: 1cc10812891d471eac59f22c1ec118f9
I20260811 06:26:27.690728 22454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:27.732688 22454 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:26:27.734016 22454 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:27.744154 22454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6758666981c04a08809baf6d7695d62c: Generated new TSK 0
I20260811 06:26:27.744942 22454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:26:27.782573 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22356
I20260811 06:26:27.783782 22427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:27.878625 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22356
2026-08-11T06:26:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2177 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22463](info): set up 2 sockets
Aug 11 06:26:27 dist-test-slave-07c2 krb5kdc[22463](info): commencing operation
krb5kdc: starting...
W20260811 06:26:29.058218 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:26:29 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:26:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:26:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:26:29.340513 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:29.677750 22478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:29.678045 22478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:29.678211 22478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:29.685360 22478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:29.685488 22478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:29.685721 22478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:29.685855 22478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:29.685943 22478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:29.695979 22478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:29.697213 22478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:29.698462 22478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:29.710147 22483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:29.710124 22484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:29.710067 22486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:29.711148 22478 server_base.cc:1034] running on GCE node
I20260811 06:26:29.711951 22478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:29.713249 22478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:29.714457 22478 hybrid_clock.cc:648] HybridClock initialized: now 1786429589714413 us; error 40 us; skew 500 ppm
I20260811 06:26:29.714790 22478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:29 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:29.721794 22478 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:29.723348 22478 webserver.cc:460] Webserver started at http://127.7.4.190:32825/ using document root <none> and password file <none>
I20260811 06:26:29.724023 22478 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:29.724169 22478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:29.724476 22478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:29.727430 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "2d3b72ecb85b4b278ef4982c26b3a61f"
format_stamp: "Formatted at 2026-08-11 06:26:29 on dist-test-slave-07c2"
server_key: "d365a0950ca8251e8265f12d4fd9674e"
server_key_iv: "43952e37f8f74ca4f8882bb967f4b7d3"
server_key_version: "encryptionkey@0"
I20260811 06:26:29.728204 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "2d3b72ecb85b4b278ef4982c26b3a61f"
format_stamp: "Formatted at 2026-08-11 06:26:29 on dist-test-slave-07c2"
server_key: "d365a0950ca8251e8265f12d4fd9674e"
server_key_iv: "43952e37f8f74ca4f8882bb967f4b7d3"
server_key_version: "encryptionkey@0"
I20260811 06:26:29.733196 22478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 06:26:29.736477 22495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:29.737699 22478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:26:29.737870 22478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "2d3b72ecb85b4b278ef4982c26b3a61f"
format_stamp: "Formatted at 2026-08-11 06:26:29 on dist-test-slave-07c2"
server_key: "d365a0950ca8251e8265f12d4fd9674e"
server_key_iv: "43952e37f8f74ca4f8882bb967f4b7d3"
server_key_version: "encryptionkey@0"
I20260811 06:26:29.738052 22478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:29.757813 22478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:29.761924 22478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:29.762193 22478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:29.780484 22478 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45995
I20260811 06:26:29.780560 22559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45995 every 8 connection(s)
I20260811 06:26:29.781756 22478 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:29.785172 22560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:29.786451 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22478
I20260811 06:26:29.786705 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:26:29.787638 7186 external_mini_cluster.cc:1428] Setting key f94f8abf26820f34a84fdb0765f34d64
I20260811 06:26:29.798158 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap starting.
Aug 11 06:26:29 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:29.801344 22560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:29.802630 22560 log.cc:824] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:29.805655 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: No bootstrap required, opened a new log
I20260811 06:26:29.809857 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:29.789814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.7.4.190:45995)
0811 06:26:29.790663 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:29.790675 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:29.790714 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:26:29.792344 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:29.792368 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:29.792723 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:29.793087 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:29.794517 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:29.795887 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:29.797243 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:29.797894 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:29.798081 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:29.802690 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:29.802697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:29.802709 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:29.802778 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:29.805951 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:29.806726 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:29.806731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:29.806737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:29.806918 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:29.807369 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:29.807372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:29.807374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:29.807845 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:29.808071 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:29.808566 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 06:26:29.808919 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":421,"threads_started":1}
I20260811 06:26:29.813928 22560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:29.814275 22560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:29.814380 22560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Initialized, Role: FOLLOWER
I20260811 06:26:29.814898 22560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:29.815124 22560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:29.815271 22560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:29.815491 22560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:29.817538 22560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:29.817999 22560 leader_election.cc:302] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d3b72ecb85b4b278ef4982c26b3a61f; no voters:
I20260811 06:26:29.818578 22560 leader_election.cc:288] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:26:29.818706 22565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:29.819444 22565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 LEADER]: Becoming Leader. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Running, Role: LEADER
I20260811 06:26:29.819967 22565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:29.820652 22560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:29.823108 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:29.823551 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:29.823961 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d3b72ecb85b4b278ef4982c26b3a61f. Latest consensus state: current_term: 1 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:29.824440 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:29.828325 22575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:29.832809 22575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:29.840692 22575 catalog_manager.cc:1347] Generated new cluster ID: 9e5b744717b544499fed90394b9b41f2
I20260811 06:26:29.840816 22575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:29.869798 22575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:26:29.871016 22575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:29.880081 22575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Generated new TSK 0
I20260811 06:26:29.880812 22575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:26:29.951071 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:30.283537 22592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:30.283807 22592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:30.283962 22592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:30.290964 22592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:30.291061 22592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:30.291260 22592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:30.300638 22592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:30.301826 22592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:30.303170 22592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:30.318272 22597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:30.318324 22602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:30.318351 22598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:30.319330 22592 server_base.cc:1034] running on GCE node
I20260811 06:26:30.319916 22592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:30.320840 22592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:30.322057 22592 hybrid_clock.cc:648] HybridClock initialized: now 1786429590322003 us; error 47 us; skew 500 ppm
I20260811 06:26:30.322381 22592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:30 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:30.325724 22592 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:30.327099 22592 webserver.cc:460] Webserver started at http://127.7.4.129:33687/ using document root <none> and password file <none>
I20260811 06:26:30.327863 22592 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:30.327996 22592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:30.328315 22592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:30.331248 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "3c4fc0a22a7726f9072f12c639db1e50"
server_key_iv: "ee8015bc86ce3f252ba246109f51731c"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.332091 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "3c4fc0a22a7726f9072f12c639db1e50"
server_key_iv: "ee8015bc86ce3f252ba246109f51731c"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.337615 22592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:26:30.341274 22610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.342774 22592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 06:26:30.342955 22592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "3c4fc0a22a7726f9072f12c639db1e50"
server_key_iv: "ee8015bc86ce3f252ba246109f51731c"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.343164 22592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:30.357684 22592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:30.362422 22592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:30.362742 22592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:30.363796 22592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:30.365382 22592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:30.365459 22592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.365620 22592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:30.365700 22592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.401444 22592 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:45811
I20260811 06:26:30.401484 22736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:45811 every 8 connection(s)
I20260811 06:26:30.402842 22592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:30.407619 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22592
I20260811 06:26:30.407886 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:26:30.408797 7186 external_mini_cluster.cc:1428] Setting key 1665ea88005d0cd32d0538ec13f1347a
Aug 11 06:26:30 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:30.428474 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:30.405688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:53699 (local address 127.7.4.190:45995)
0811 06:26:30.406330 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:30.406336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:30.407260 (+ 924us) server_negotiation.cc:407] Connection header received
0811 06:26:30.410010 (+ 2750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:30.410015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:30.410121 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:30.410250 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:30.412416 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.413175 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.415937 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.416366 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.416476 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:30.421193 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:30.421198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:30.421201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:30.421237 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:30.424979 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:30.425845 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:30.425851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:30.425855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:30.425996 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:30.426495 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:30.426506 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:30.426508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:30.426801 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:30.426970 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:30.427831 (+ 861us) server_negotiation.cc:299] Negotiation successful
0811 06:26:30.428102 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":305,"threads_started":1}
I20260811 06:26:30.429625 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:30.406203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:53699)
0811 06:26:30.406975 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:30.407062 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:30.409703 (+ 2641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:30.410526 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:30.410536 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:30.411257 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:30.412196 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:30.412229 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.413372 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.413381 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:30.415752 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:30.415775 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.416628 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.416638 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:30.417018 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:30.417822 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:30.417863 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:30.420913 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:30.425197 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:30.425205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:30.425220 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:30.425689 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:30.426125 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:30.426130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:30.426133 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:30.426354 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:30.426948 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:30.426957 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:30.427393 (+ 436us) client_negotiation.cc:769] Sending connection context
0811 06:26:30.427677 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:26:30.428267 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":325,"threads_started":1}
I20260811 06:26:30.431371 22739 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:30.431751 22739 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:30.432869 22739 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:30.435518 22518 ts_manager.cc:194] Registered new tserver with Master: efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811)
I20260811 06:26:30.437055 22518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:53699
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:26:30.483182 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:30.833313 22747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:30.833629 22747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:30.833781 22747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:30.841274 22747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:30.841384 22747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:30.841557 22747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:30.851473 22747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:30.852694 22747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:30.854045 22747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:30.867274 22752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:30.867187 22753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:30.867210 22755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:30.868263 22747 server_base.cc:1034] running on GCE node
I20260811 06:26:30.868880 22747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:30.869810 22747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:30.871033 22747 hybrid_clock.cc:648] HybridClock initialized: now 1786429590870986 us; error 65 us; skew 500 ppm
I20260811 06:26:30.871411 22747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:30 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:30.874608 22747 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:30.876192 22747 webserver.cc:460] Webserver started at http://127.7.4.130:38777/ using document root <none> and password file <none>
I20260811 06:26:30.876874 22747 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:30.877017 22747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:30.877323 22747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:30.880256 22747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "debb097781464bb49be8bfa30f186a2e"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "49bf5ce1b9e973faae33cce8e010a02a"
server_key_iv: "e5bbd7b5504e1d46fc057bbc1e888467"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.881014 22747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "debb097781464bb49be8bfa30f186a2e"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "49bf5ce1b9e973faae33cce8e010a02a"
server_key_iv: "e5bbd7b5504e1d46fc057bbc1e888467"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.886161 22747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:26:30.889488 22762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.890707 22747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:26:30.890931 22747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "debb097781464bb49be8bfa30f186a2e"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "49bf5ce1b9e973faae33cce8e010a02a"
server_key_iv: "e5bbd7b5504e1d46fc057bbc1e888467"
server_key_version: "encryptionkey@0"
I20260811 06:26:30.891153 22747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:30.914942 22747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:30.918846 22747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:30.919142 22747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:30.920224 22747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:30.921640 22747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:30.921749 22747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.921887 22747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:30.921957 22747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:30.956364 22747 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:33883
I20260811 06:26:30.956450 22886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:33883 every 8 connection(s)
I20260811 06:26:30.957659 22747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:30.965376 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22747
I20260811 06:26:30.965562 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:26:30.966090 7186 external_mini_cluster.cc:1428] Setting key 639576cb93c359d08419e6c2ca3a8a00
Aug 11 06:26:30 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:30.981141 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:30.960238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38423 (local address 127.7.4.190:45995)
0811 06:26:30.960712 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:30.960719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:30.961658 (+ 939us) server_negotiation.cc:407] Connection header received
0811 06:26:30.963434 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:30.963439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:30.963506 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:30.963604 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:30.965006 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.965799 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.968134 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.968770 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.968907 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:30.972327 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:30.972335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:30.972343 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:30.972389 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:30.975995 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:30.978354 (+ 2359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:30.978363 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:30.978369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:30.978587 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:30.979020 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:30.979027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:30.979033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:30.979449 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:30.979606 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:30.980267 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 06:26:30.980525 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":214,"threads_started":1}
I20260811 06:26:30.981535 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:30.960782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:38423)
0811 06:26:30.961488 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:30.961525 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:30.963133 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:30.963797 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:30.963805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:30.964311 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:30.964842 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:30.964859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.965946 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.965953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:30.967915 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:30.967934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:30.968983 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:30.968988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:30.969191 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:30.969869 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:30.969891 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:30.972105 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:30.976181 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:30.976196 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:30.976215 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:30.976919 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:30.978711 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:30.978715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:30.978718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:30.978899 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:30.979584 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:30.979590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:30.979897 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 06:26:30.980117 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 06:26:30.980490 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":277,"threads_started":1}
I20260811 06:26:30.983004 22887 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:30.983424 22887 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:30.984138 22887 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:30.985556 22518 ts_manager.cc:194] Registered new tserver with Master: debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
I20260811 06:26:30.986356 22518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:38423
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:26:31.030784 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:31.379644 22895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:31.380085 22895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:31.380281 22895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:31.390707 22895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:31.390818 22895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:31.390944 22895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:31.401355 22895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:31.402948 22895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:31.404685 22895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:31.417876 22904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:31.417968 22901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:31.417941 22902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:31.418946 22895 server_base.cc:1034] running on GCE node
I20260811 06:26:31.419559 22895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:31.420501 22895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:31.421759 22895 hybrid_clock.cc:648] HybridClock initialized: now 1786429591421721 us; error 44 us; skew 500 ppm
I20260811 06:26:31.422062 22895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:31 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:31.425083 22895 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:31.426566 22895 webserver.cc:460] Webserver started at http://127.7.4.131:37695/ using document root <none> and password file <none>
I20260811 06:26:31.427248 22895 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:31.427440 22895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:31.427817 22895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:31.430852 22895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "f7c90a403c904d08999354cb946c1480"
format_stamp: "Formatted at 2026-08-11 06:26:31 on dist-test-slave-07c2"
server_key: "16826d7e81cf8c095266b6ddf1cfc0dd"
server_key_iv: "c7b23283ee391c5beb17ae04d2971119"
server_key_version: "encryptionkey@0"
I20260811 06:26:31.431669 22895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "f7c90a403c904d08999354cb946c1480"
format_stamp: "Formatted at 2026-08-11 06:26:31 on dist-test-slave-07c2"
server_key: "16826d7e81cf8c095266b6ddf1cfc0dd"
server_key_iv: "c7b23283ee391c5beb17ae04d2971119"
server_key_version: "encryptionkey@0"
I20260811 06:26:31.436606 22895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:26:31.439980 22912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:31.441144 22739 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:31.441371 22895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:26:31.441530 22895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "f7c90a403c904d08999354cb946c1480"
format_stamp: "Formatted at 2026-08-11 06:26:31 on dist-test-slave-07c2"
server_key: "16826d7e81cf8c095266b6ddf1cfc0dd"
server_key_iv: "c7b23283ee391c5beb17ae04d2971119"
server_key_version: "encryptionkey@0"
I20260811 06:26:31.441754 22895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:31.463845 22895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:31.467617 22895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:31.467909 22895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:31.468845 22895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:31.470124 22895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:31.470202 22895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:31.470373 22895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:31.470424 22895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:31.504858 22895 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:44273
I20260811 06:26:31.504938 23037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:44273 every 8 connection(s)
I20260811 06:26:31.506166 22895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:31.508560 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 22895
I20260811 06:26:31.508859 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:26:31.509606 7186 external_mini_cluster.cc:1428] Setting key 3ca84754abe5a623784c9cf7dbe5eaf7
Aug 11 06:26:31 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:31.527580 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.509226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:54173 (local address 127.7.4.190:45995)
0811 06:26:31.509787 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.509793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.510808 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 06:26:31.512931 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.512936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.513033 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.513159 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:31.514625 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.515501 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.517026 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.517441 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.517577 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.521346 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:31.521351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:31.521354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:31.521386 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:31.524315 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.525251 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.525266 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.525269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.525404 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.525855 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.525861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.525864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.526136 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:31.526302 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.526945 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.527173 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":287,"threads_started":1}
I20260811 06:26:31.528548 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.509741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:54173)
0811 06:26:31.510638 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.510700 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.512659 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.513411 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.513419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.513928 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:31.514448 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.514464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.515665 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.515669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.516858 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.516869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.517674 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.517682 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.517922 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.518737 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:31.518760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:31.521124 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:31.524459 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:31.524468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:31.524489 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:31.525109 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:31.525520 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:31.525525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:31.525529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:31.525737 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:31.526267 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:31.526275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:31.526594 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.526850 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.527352 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":377,"threads_started":1}
I20260811 06:26:31.529876 23038 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:31.530135 23038 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:31.530794 23038 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:31.532147 22518 ts_manager.cc:194] Registered new tserver with Master: f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:31.532946 22518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:54173
I20260811 06:26:31.537559 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:26:31.555372 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.542344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.7.4.190:45995)
0811 06:26:31.542628 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.542634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.542955 (+ 321us) server_negotiation.cc:407] Connection header received
0811 06:26:31.543123 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.543127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.543190 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.543276 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:31.544751 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.545603 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.547030 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.547554 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.547716 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.550218 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:31.550222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:31.550226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:31.550258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:31.552915 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.553536 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.553540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.553543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.553684 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.554016 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.554019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.554021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.554318 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:31.554500 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.554780 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.555037 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:26:31.559558 22518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:26:31.566416 22518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:26:31.617090 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.586747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59158)
0811 06:26:31.601862 (+ 15115us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.601913 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.602104 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.608426 (+ 6322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.608433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.608472 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:31.608828 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.608839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.610852 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.610858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.613535 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.613553 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.615818 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.615824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.616086 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.616146 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.616282 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.616481 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14830,"mutex_wait_us":69,"thread_start_us":15548,"threads_started":1}
I20260811 06:26:31.618247 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.587086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.7.4.129:45811)
0811 06:26:31.607396 (+ 20310us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.607408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.607443 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:26:31.607579 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.607588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.607871 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.608119 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:31.609042 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.610712 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.613711 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.615223 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.615424 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.615668 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.617585 (+ 1917us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.617875 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20036,"thread_start_us":21242,"threads_started":1}
I20260811 06:26:31.618863 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.587677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.7.4.130:33883)
0811 06:26:31.610650 (+ 22973us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.610659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.610686 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:26:31.610798 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.610805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.610990 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.611197 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:31.612011 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.613272 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.615997 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.617601 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.617870 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.618088 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.618333 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.618510 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22815,"thread_start_us":23635,"threads_started":1}
I20260811 06:26:31.618872 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.587043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33883 (local address 127.0.0.1:60210)
0811 06:26:31.609985 (+ 22942us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.610004 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.610150 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.611176 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.611181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.611208 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:31.611766 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.611787 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.613404 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.613412 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.615804 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.615816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.617912 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.617918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.618104 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.618144 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.618272 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.618404 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22809,"mutex_wait_us":62,"thread_start_us":22652,"threads_started":1}
I20260811 06:26:31.619474 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.587369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:44273 (local address 127.0.0.1:40098)
0811 06:26:31.610435 (+ 23066us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.610450 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.610603 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.611565 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.611571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.611618 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:31.612110 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.612123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.614043 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.614048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.616128 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.616142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.618268 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.618277 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.618464 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.618500 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.618590 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.618728 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22956,"mutex_wait_us":67,"thread_start_us":22797,"threads_started":1}
I20260811 06:26:31.619551 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.587983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.7.4.131:44273)
0811 06:26:31.609532 (+ 21549us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.609541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.610559 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 06:26:31.610795 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.610804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.611046 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.611240 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:31.612344 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.613904 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.616302 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.617789 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.617927 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.618152 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.618979 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.619163 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21371,"thread_start_us":21207,"threads_started":1}
I20260811 06:26:31.622305 22665 tablet_service.cc:1467] Processing CreateTablet for tablet 9b135403479a4c0a927547278e43f2ac (DEFAULT_TABLE table=test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:31.622748 22816 tablet_service.cc:1467] Processing CreateTablet for tablet 9b135403479a4c0a927547278e43f2ac (DEFAULT_TABLE table=test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:31.623023 22963 tablet_service.cc:1467] Processing CreateTablet for tablet 9b135403479a4c0a927547278e43f2ac (DEFAULT_TABLE table=test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:31.623970 22665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b135403479a4c0a927547278e43f2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:31.624477 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b135403479a4c0a927547278e43f2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:31.624750 22963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b135403479a4c0a927547278e43f2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:31.635627 23061 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Bootstrap starting.
I20260811 06:26:31.638796 23061 tablet_bootstrap.cc:654] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:31.639173 23062 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Bootstrap starting.
I20260811 06:26:31.640025 23065 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Bootstrap starting.
I20260811 06:26:31.640384 23061 log.cc:824] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:31.643198 23061 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: No bootstrap required, opened a new log
I20260811 06:26:31.643509 23061 ts_tablet_manager.cc:1397] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260811 06:26:31.644150 23062 tablet_bootstrap.cc:654] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:31.644150 23065 tablet_bootstrap.cc:654] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:31.645434 23062 log.cc:824] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:31.645435 23065 log.cc:824] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:31.648818 23065 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: No bootstrap required, opened a new log
I20260811 06:26:31.649123 23065 ts_tablet_manager.cc:1397] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:26:31.649809 23062 tablet_bootstrap.cc:492] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: No bootstrap required, opened a new log
I20260811 06:26:31.650069 23062 ts_tablet_manager.cc:1397] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 06:26:31.652072 23061 raft_consensus.cc:348] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.652444 23061 raft_consensus.cc:374] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:31.652581 23061 raft_consensus.cc:729] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbf8dd867754106a65a9c6eb8d0d35c, State: Initialized, Role: FOLLOWER
I20260811 06:26:31.653190 23061 consensus_queue.cc:260] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.659855 23065 raft_consensus.cc:348] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.660313 23065 raft_consensus.cc:374] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:31.660485 23065 raft_consensus.cc:729] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c90a403c904d08999354cb946c1480, State: Initialized, Role: FOLLOWER
I20260811 06:26:31.660926 23062 raft_consensus.cc:348] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.661384 23062 raft_consensus.cc:374] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:31.661516 23062 raft_consensus.cc:729] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: debb097781464bb49be8bfa30f186a2e, State: Initialized, Role: FOLLOWER
I20260811 06:26:31.661348 23065 consensus_queue.cc:260] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.661801 23061 ts_tablet_manager.cc:1428] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Time spent starting tablet: real 0.018s user 0.006s sys 0.011s
I20260811 06:26:31.663561 23062 consensus_queue.cc:260] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.664755 22887 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:31.665503 23062 ts_tablet_manager.cc:1428] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260811 06:26:31.666600 23038 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:31.667246 23065 ts_tablet_manager.cc:1428] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260811 06:26:31.698711 23073 raft_consensus.cc:482] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:31.698961 23073 raft_consensus.cc:504] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.700543 23073 leader_election.cc:288] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811), f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:31.707397 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.701084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:56839 (local address 127.7.4.131:44273)
0811 06:26:31.701328 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.701338 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.701926 (+ 588us) server_negotiation.cc:407] Connection header received
0811 06:26:31.702115 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.702119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.702175 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.702274 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:31.702887 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.703731 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.705645 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.706405 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.706500 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.706573 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.706990 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.707133 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:26:31.707494 23075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.701000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:44273 (local address 127.7.4.130:56839)
0811 06:26:31.701792 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.701806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.701966 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.702295 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.702299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.702325 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:31.702726 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.702735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.703869 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.703874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.705492 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.705505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.706634 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.706641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.706819 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.706846 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.706919 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.707081 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"spinlock_wait_cycles":896,"thread_start_us":501,"threads_started":1}
I20260811 06:26:31.707517 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.700850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:34333 (local address 127.7.4.129:45811)
0811 06:26:31.701107 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.701116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.701882 (+ 766us) server_negotiation.cc:407] Connection header received
0811 06:26:31.702094 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.702101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.702172 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.702272 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:31.703006 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.703761 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.705707 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.706504 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.706655 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.706756 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.707113 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.707247 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:26:31.708220 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.700805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.7.4.130:34333)
0811 06:26:31.701729 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:31.701751 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:31.701925 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:31.702296 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:31.702300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:31.702394 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:31.702849 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.702861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.703920 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.703927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.705518 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:31.705530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.706754 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.706759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:31.706911 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.706952 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:26:31.707653 (+ 701us) client_negotiation.cc:240] Negotiation successful
0811 06:26:31.707785 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":44,"spinlock_wait_cycles":23936,"thread_start_us":515,"threads_started":1}
I20260811 06:26:31.708606 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b135403479a4c0a927547278e43f2ac" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7c90a403c904d08999354cb946c1480" is_pre_election: true
I20260811 06:26:31.709177 22985 raft_consensus.cc:2454] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 0.
I20260811 06:26:31.709236 22688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b135403479a4c0a927547278e43f2ac" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" is_pre_election: true
W20260811 06:26:31.709554 22888 tablet.cc:2369] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:31.709821 22688 raft_consensus.cc:2454] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 0.
I20260811 06:26:31.709983 22764 leader_election.cc:302] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480; no voters:
I20260811 06:26:31.710474 23073 raft_consensus.cc:2781] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:26:31.710630 23073 raft_consensus.cc:482] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:26:31.710745 23073 raft_consensus.cc:3037] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:31.713227 23073 raft_consensus.cc:504] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.713881 23073 leader_election.cc:288] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 election: Requested vote from peers efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811), f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:31.714260 22688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b135403479a4c0a927547278e43f2ac" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
I20260811 06:26:31.714423 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b135403479a4c0a927547278e43f2ac" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7c90a403c904d08999354cb946c1480"
I20260811 06:26:31.714489 22688 raft_consensus.cc:3037] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:31.714648 22985 raft_consensus.cc:3037] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:31.717856 22985 raft_consensus.cc:2454] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 1.
I20260811 06:26:31.717854 22688 raft_consensus.cc:2454] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 1.
I20260811 06:26:31.718391 22767 leader_election.cc:302] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: debb097781464bb49be8bfa30f186a2e, efbf8dd867754106a65a9c6eb8d0d35c; no voters:
I20260811 06:26:31.718780 23073 raft_consensus.cc:2781] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:31.719375 23073 raft_consensus.cc:686] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 1 LEADER]: Becoming Leader. State: Replica: debb097781464bb49be8bfa30f186a2e, State: Running, Role: LEADER
I20260811 06:26:31.719846 23073 consensus_queue.cc:237] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } }
I20260811 06:26:31.724741 22518 catalog_manager.cc:5495] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e reported cstate change: term changed from 0 to 1, leader changed from <none> to debb097781464bb49be8bfa30f186a2e (127.7.4.130). New cstate: current_term: 1 leader_uuid: "debb097781464bb49be8bfa30f186a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } health_report { overall_health: UNKNOWN } } }
I20260811 06:26:31.753422 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.747824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.7.4.130:33883)
0811 06:26:31.748027 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.748035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.748055 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:26:31.748369 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.748373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.748449 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.748603 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:26:31.749007 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.749746 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.751309 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.751712 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.751852 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.752345 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:26:31.752851 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:26:31.753011 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.753072 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.753154 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 06:26:31.758162 23039 tablet.cc:2369] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:31.761485 22985 raft_consensus.cc:1261] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Refusing update from remote peer debb097781464bb49be8bfa30f186a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:31.761511 22688 raft_consensus.cc:1261] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Refusing update from remote peer debb097781464bb49be8bfa30f186a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:31.762445 23076 consensus_queue.cc:1035] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:31.762777 23073 consensus_queue.cc:1035] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:31.776991 23081 mvcc.cc:205] Tried to move back new op lower bound from 7317215607845203968 to 7317215607688536064. Current Snapshot: MvccSnapshot[applied={T|T < 7317215607845203968}]
I20260811 06:26:31.792280 22518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:31.792611 22518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:31.797603 22518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Sending DeleteTablet for 3 replicas of tablet 9b135403479a4c0a927547278e43f2ac
I20260811 06:26:31.799296 22665 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b135403479a4c0a927547278e43f2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:31 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:59158
I20260811 06:26:31.799525 22963 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b135403479a4c0a927547278e43f2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:31 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:40098
I20260811 06:26:31.799649 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22592
I20260811 06:26:31.798904 22816 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b135403479a4c0a927547278e43f2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:31 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60210
I20260811 06:26:31.817719 23089 tablet_replica.cc:330] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: stopping tablet replica
I20260811 06:26:31.818210 23089 raft_consensus.cc:2229] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:31.818751 23089 raft_consensus.cc:2258] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:31.823336 23089 ts_tablet_manager.cc:1905] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:31.825598 23093 tablet_replica.cc:330] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: stopping tablet replica
I20260811 06:26:31.826145 23093 raft_consensus.cc:2229] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:26:31.826975 23093 raft_consensus.cc:2258] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:31.829456 23090 tablet_replica.cc:330] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: stopping tablet replica
I20260811 06:26:31.829879 23090 raft_consensus.cc:2229] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:31.830380 23090 raft_consensus.cc:2258] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:31.831593 23093 ts_tablet_manager.cc:1905] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:31.832270 23089 ts_tablet_manager.cc:1918] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:31.832451 23089 log.cc:1196] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/9b135403479a4c0a927547278e43f2ac
I20260811 06:26:31.833055 23089 ts_tablet_manager.cc:1939] T 9b135403479a4c0a927547278e43f2ac P efbf8dd867754106a65a9c6eb8d0d35c: Deleting consensus metadata
I20260811 06:26:31.834918 22502 catalog_manager.cc:4841] TS efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811): tablet 9b135403479a4c0a927547278e43f2ac (table test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]) successfully deleted
I20260811 06:26:31.837152 23090 ts_tablet_manager.cc:1905] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:31.839932 23093 ts_tablet_manager.cc:1918] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:31.840197 23093 log.cc:1196] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/9b135403479a4c0a927547278e43f2ac
Aug 11 06:26:31 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:26:31.840927 23093 ts_tablet_manager.cc:1939] T 9b135403479a4c0a927547278e43f2ac P debb097781464bb49be8bfa30f186a2e: Deleting consensus metadata
I20260811 06:26:31.842929 22501 catalog_manager.cc:4841] TS debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): tablet 9b135403479a4c0a927547278e43f2ac (table test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]) successfully deleted
I20260811 06:26:31.844180 23090 ts_tablet_manager.cc:1918] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:31.844334 23090 log.cc:1196] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/9b135403479a4c0a927547278e43f2ac
I20260811 06:26:31.844875 23090 ts_tablet_manager.cc:1939] T 9b135403479a4c0a927547278e43f2ac P f7c90a403c904d08999354cb946c1480: Deleting consensus metadata
I20260811 06:26:31.846450 22498 catalog_manager.cc:4841] TS f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): tablet 9b135403479a4c0a927547278e43f2ac (table test-table [id=e0e982c59c07455f8b7bbcfbbab39f59]) successfully deleted
I20260811 06:26:31.849244 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.800964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.7.4.129:45811)
0811 06:26:31.801184 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.801193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.824936 (+ 23743us) server_negotiation.cc:407] Connection header received
0811 06:26:31.827959 (+ 3023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.827969 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.828083 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.828176 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:31.832363 (+ 4187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.833562 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.835579 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.836200 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.836359 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.842343 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:31.842357 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:31.842373 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:31.842412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:31.845861 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.846945 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.846957 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.846960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.847132 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.847595 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.847600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.847604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.847985 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:31.848303 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.848642 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.848887 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:26:31.849926 22732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:31.939303 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22592
I20260811 06:26:31.962966 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22747
Aug 11 06:26:31 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:26:31.980901 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:31.963936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.7.4.130:33883)
0811 06:26:31.964119 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:31.964125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:31.964285 (+ 160us) server_negotiation.cc:407] Connection header received
0811 06:26:31.964503 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:31.964508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:31.964588 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:31.964710 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:31.965962 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.966820 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.968245 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:31.968722 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:31.968871 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:31.973184 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:31.973195 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:31.973208 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:31.973247 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:31.976382 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.977086 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.977100 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.977102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.977247 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:31.977666 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:31.977671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:31.977677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:31.978028 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:31.978408 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:31.979915 (+ 1507us) server_negotiation.cc:299] Negotiation successful
0811 06:26:31.980278 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:26:31.981952 22881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:32.097095 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22747
I20260811 06:26:32.121165 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22895
Aug 11 06:26:32 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:26:32.140569 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:32.126472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.7.4.131:44273)
0811 06:26:32.126684 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:32.126689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:32.126710 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:26:32.126856 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:32.126860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:32.126934 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:32.127017 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:32.128419 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:32.129228 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:32.130413 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:32.130779 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:32.130869 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:32.135080 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:32.135095 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:32.135111 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:32.135146 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:32.137852 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:32.138525 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:32.138533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:32.138535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:32.138691 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:32.139079 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:32.139082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:32.139084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:32.139529 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:32.139828 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:32.140138 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 06:26:32.140314 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:26:32.141240 23033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:32.230641 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22895
I20260811 06:26:32.253309 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 22478
I20260811 06:26:32.254547 22555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:32.366495 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 22478
I20260811 06:26:32.393839 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--webserver_interface=127.7.4.190
--webserver_port=32825
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:32.739639 23098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:32.739987 23098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:32.740154 23098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:32.747366 23098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:32.747465 23098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:32.747511 23098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:32.747568 23098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:32.747611 23098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:32.757303 23098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:32.758641 23098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:32.760035 23098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:32.785590 23104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:32.785694 23107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:32.785591 23105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:32.786500 23098 server_base.cc:1034] running on GCE node
I20260811 06:26:32.787494 23098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:32.789086 23098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:32.790310 23098 hybrid_clock.cc:648] HybridClock initialized: now 1786429592790264 us; error 27 us; skew 500 ppm
I20260811 06:26:32.790745 23098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:32 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429592, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:32.794500 23098 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:32.796227 23098 webserver.cc:460] Webserver started at http://127.7.4.190:32825/ using document root <none> and password file <none>
I20260811 06:26:32.797034 23098 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:32.797204 23098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:32.803411 23098 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:26:32.806239 23116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:32.807598 23098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:26:32.807796 23098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "2d3b72ecb85b4b278ef4982c26b3a61f"
format_stamp: "Formatted at 2026-08-11 06:26:29 on dist-test-slave-07c2"
server_key: "d365a0950ca8251e8265f12d4fd9674e"
server_key_iv: "43952e37f8f74ca4f8882bb967f4b7d3"
server_key_version: "encryptionkey@0"
I20260811 06:26:32.808898 23098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:32.829741 23098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:32.833825 23098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:32.834128 23098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:32.853281 23098 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45995
I20260811 06:26:32.853310 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45995 every 8 connection(s)
I20260811 06:26:32.854722 23098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:32.861589 23177 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:26:32.862287 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23098
I20260811 06:26:32.863098 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:45811
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=33687
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:32.864902 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap starting.
I20260811 06:26:32.887266 23177 log.cc:824] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:32.904117 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:26:32.904747 23177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap complete.
I20260811 06:26:32.918428 23177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:32.919678 23177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Initialized, Role: FOLLOWER
I20260811 06:26:32.920392 23177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:32.920691 23177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:32.920826 23177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:32.920998 23177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:26:32.926537 23177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:32.927172 23177 leader_election.cc:302] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d3b72ecb85b4b278ef4982c26b3a61f; no voters:
I20260811 06:26:32.927922 23177 leader_election.cc:288] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:26:32.928103 23181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:26:32.929368 23181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 LEADER]: Becoming Leader. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Running, Role: LEADER
I20260811 06:26:32.929880 23181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:32.930187 23177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:32.932555 23184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d3b72ecb85b4b278ef4982c26b3a61f. Latest consensus state: current_term: 2 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:32.932533 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:32.932960 23184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:32.932956 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:32.935375 23188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:32.946514 23188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:32.947926 23188 catalog_manager.cc:1259] Loaded cluster ID: 9e5b744717b544499fed90394b9b41f2
I20260811 06:26:32.948026 23188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:32.951426 23188 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 06:26:32.969344 23188 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 06:26:32.979435 23188 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:32.981144 23188 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Loaded TSK: 0
I20260811 06:26:32.982561 23188 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:26:33.237831 23179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:33.238129 23179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:33.238284 23179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:33.245558 23179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:33.245709 23179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:33.245878 23179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:33.255957 23179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:45811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=33687
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:33.257238 23179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:33.258651 23179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:33.273301 23208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:33.273327 23207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:33.273764 23210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:33.274580 23179 server_base.cc:1034] running on GCE node
I20260811 06:26:33.275199 23179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:33.276196 23179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:33.277438 23179 hybrid_clock.cc:648] HybridClock initialized: now 1786429593277380 us; error 47 us; skew 500 ppm
I20260811 06:26:33.277743 23179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:33 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429593, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:33.280920 23179 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:33.282435 23179 webserver.cc:460] Webserver started at http://127.7.4.129:33687/ using document root <none> and password file <none>
I20260811 06:26:33.283115 23179 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:33.283286 23179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:33.288615 23179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:26:33.291445 23217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.292855 23179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:33.293015 23179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "3c4fc0a22a7726f9072f12c639db1e50"
server_key_iv: "ee8015bc86ce3f252ba246109f51731c"
server_key_version: "encryptionkey@0"
I20260811 06:26:33.293994 23179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:33.328679 23179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:33.333145 23179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:33.333439 23179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:33.334432 23179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:33.335736 23179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:33.335856 23179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.336000 23179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:33.336094 23179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.371167 23179 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:45811
I20260811 06:26:33.371268 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:45811 every 8 connection(s)
I20260811 06:26:33.372725 23179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:33.377936 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23179
I20260811 06:26:33.378767 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:33883
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=38777
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:33 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429593, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:33.400349 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:33.375984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:37143)
0811 06:26:33.376810 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:33.376854 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:33.379315 (+ 2461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:33.380835 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:33.380847 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:33.381654 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:33.382445 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:33.382471 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.384171 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.384179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:33.386290 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:33.386308 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.387358 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.387369 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:33.387671 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:33.388401 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:33.388435 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:33.391339 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:33.396096 (+ 4757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:33.396102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:33.396112 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:33.396527 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:33.397077 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:33.397081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:33.397083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:33.397337 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:33.398253 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:33.398258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:33.398581 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 06:26:33.398823 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 06:26:33.399172 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":351,"threads_started":1}
I20260811 06:26:33.400933 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:33.375657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:37143 (local address 127.7.4.190:45995)
0811 06:26:33.376321 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:33.376336 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:33.376983 (+ 647us) server_negotiation.cc:407] Connection header received
0811 06:26:33.379862 (+ 2879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:33.379874 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:33.380294 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:33.380624 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:33.382683 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.383997 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.386448 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.387086 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.387220 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:33.391655 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:33.391667 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:33.391688 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:33.391741 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:33.395942 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:33.396676 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:33.396689 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:33.396699 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:33.396943 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:33.397448 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:33.397456 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:33.397462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:33.398118 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:33.398341 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:33.399082 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 06:26:33.399647 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":256,"threads_started":1}
I20260811 06:26:33.403542 23344 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:33.403836 23344 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:33.404546 23344 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:33.406764 23134 ts_manager.cc:194] Registered new tserver with Master: efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811)
I20260811 06:26:33.408246 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:37143
W20260811 06:26:33.732349 23348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:33.732666 23348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:33.732846 23348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:33.740940 23348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:33.741075 23348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:33.741273 23348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:33.752034 23348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:33883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=38777
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:33.753801 23348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:33.755200 23348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:33.768996 23358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:33.769181 23355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:33.769191 23356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:33.770037 23348 server_base.cc:1034] running on GCE node
I20260811 06:26:33.770720 23348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:33.771719 23348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:33.772948 23348 hybrid_clock.cc:648] HybridClock initialized: now 1786429593772902 us; error 64 us; skew 500 ppm
I20260811 06:26:33.773267 23348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:33 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429593, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:33.776525 23348 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:33.777925 23348 webserver.cc:460] Webserver started at http://127.7.4.130:38777/ using document root <none> and password file <none>
I20260811 06:26:33.778625 23348 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:33.778797 23348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:33.784297 23348 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:26:33.787052 23366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.788419 23348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:26:33.788612 23348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "debb097781464bb49be8bfa30f186a2e"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "49bf5ce1b9e973faae33cce8e010a02a"
server_key_iv: "e5bbd7b5504e1d46fc057bbc1e888467"
server_key_version: "encryptionkey@0"
I20260811 06:26:33.789606 23348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:33.817739 23348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:33.821918 23348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:33.822207 23348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:33.823210 23348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:33.824513 23348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:33.824625 23348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.824761 23348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:33.824854 23348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:33.861152 23348 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:33883
I20260811 06:26:33.861198 23497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:33883 every 8 connection(s)
I20260811 06:26:33.862813 23348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:33.872305 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23348
I20260811 06:26:33.873026 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:44273
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=37695
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:26:33 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429593, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:33.883268 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:33.865844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38083 (local address 127.7.4.190:45995)
0811 06:26:33.866017 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:33.866022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:33.867399 (+ 1377us) server_negotiation.cc:407] Connection header received
0811 06:26:33.869243 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:33.869249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:33.869370 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:33.869481 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:33.870948 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.871732 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.873465 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.873906 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.874040 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:33.877680 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:33.877685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:33.877689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:33.877720 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:33.880315 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:33.881014 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:33.881022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:33.881026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:33.881170 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:33.881594 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:33.881600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:33.881603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:33.881877 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:33.882047 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:33.882697 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 06:26:33.882927 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:26:33.884370 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:33.866305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:38083)
0811 06:26:33.867189 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:33.867231 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:33.868977 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:33.869723 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:33.869731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:33.870241 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:33.870778 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:33.870795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.871905 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.871917 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:33.873312 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:33.873324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:33.874132 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:33.874139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:33.874372 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:33.875033 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:33.875056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:33.877461 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:33.880464 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:33.880472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:33.880486 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:33.880894 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:33.881287 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:33.881292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:33.881295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:33.881480 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:33.882022 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:33.882029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:33.882335 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 06:26:33.882592 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 06:26:33.883096 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":371,"threads_started":1}
I20260811 06:26:33.885846 23499 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:33.886147 23499 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:33.886845 23499 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:33.888340 23134 ts_manager.cc:194] Registered new tserver with Master: debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
I20260811 06:26:33.889154 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:38083
W20260811 06:26:34.209069 23502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:34.209378 23502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:34.209550 23502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:34.216825 23502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:34.216974 23502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:34.217154 23502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:34.226608 23502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:44273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=37695
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:34.227965 23502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:34.229290 23502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:34.242455 23510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:34.242517 23512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:34.242372 23509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:34.243525 23502 server_base.cc:1034] running on GCE node
I20260811 06:26:34.244140 23502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:34.245091 23502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:34.246322 23502 hybrid_clock.cc:648] HybridClock initialized: now 1786429594246278 us; error 54 us; skew 500 ppm
I20260811 06:26:34.246626 23502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:34 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429594, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:34.249703 23502 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:34.251104 23502 webserver.cc:460] Webserver started at http://127.7.4.131:37695/ using document root <none> and password file <none>
I20260811 06:26:34.251819 23502 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:34.251982 23502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:34.257040 23502 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:26:34.259701 23519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:34.260972 23502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:26:34.261143 23502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "f7c90a403c904d08999354cb946c1480"
format_stamp: "Formatted at 2026-08-11 06:26:31 on dist-test-slave-07c2"
server_key: "16826d7e81cf8c095266b6ddf1cfc0dd"
server_key_iv: "c7b23283ee391c5beb17ae04d2971119"
server_key_version: "encryptionkey@0"
I20260811 06:26:34.262086 23502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:34.273779 23502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:34.277487 23502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:34.277771 23502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:34.278687 23502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:34.280004 23502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:34.280138 23502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:34.280272 23502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:34.280361 23502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:34.313221 23502 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:44273
I20260811 06:26:34.313297 23642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:44273 every 8 connection(s)
I20260811 06:26:34.314736 23502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:34.322103 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23502
Aug 11 06:26:34 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429594, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:34.336848 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.317166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:43043 (local address 127.7.4.190:45995)
0811 06:26:34.317349 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.317354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.318420 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 06:26:34.320413 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.320419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.320514 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.320627 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:34.322041 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.323021 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.325957 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.326403 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.326497 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.330346 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:34.330351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:34.330354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:34.330384 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:34.333466 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.334410 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.334413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.334416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.334533 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.335023 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.335027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.335028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.335358 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:34.335527 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.336218 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.336470 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:26:34.337559 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.317548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:43043)
0811 06:26:34.318256 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.318293 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.320151 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.320853 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.320861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.321352 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:34.321866 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.321882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.323217 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.323227 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.325778 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.325800 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.326620 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.326630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.326916 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.327632 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:34.327653 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:34.330134 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:34.333614 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:34.333623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:34.333638 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:34.334284 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:34.334641 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:34.334647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:34.334650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:34.334907 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:34.335489 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:34.335495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:34.335773 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.336046 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.336460 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":286,"threads_started":1}
I20260811 06:26:34.338987 23643 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45995
I20260811 06:26:34.339354 23643 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:34.340245 23643 heartbeater.cc:511] Master 127.7.4.190:45995 requested a full tablet report, sending...
I20260811 06:26:34.341602 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.322996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.7.4.190:45995)
0811 06:26:34.326391 (+ 3395us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.326395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.326409 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:26:34.326447 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.326475 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.326552 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.326672 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:34.328428 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.329371 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.331178 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.332003 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.332156 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.335349 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:34.335354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:34.335359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:34.335391 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:34.338598 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.339462 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.339469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.339475 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.339616 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.340021 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.340026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.340029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.340400 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:34.340655 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.340926 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.341162 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3314,"thread_start_us":3148,"threads_started":1}
I20260811 06:26:34.342020 23134 ts_manager.cc:194] Registered new tserver with Master: f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:34.343009 23134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:43043
I20260811 06:26:34.344161 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:26:34.350266 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.345031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.7.4.190:45995)
0811 06:26:34.345221 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.345229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.345666 (+ 437us) server_negotiation.cc:407] Connection header received
0811 06:26:34.345835 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.345838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.345945 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.346053 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:26:34.346430 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.347066 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.348519 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.348938 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.349027 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.349440 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:26:34.349837 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:26:34.349946 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.350030 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.350083 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:26:34.351059 23133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:26:34.358304 23133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:26:34.384967 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.369993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59168)
0811 06:26:34.372841 (+ 2848us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.372858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.372957 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.376858 (+ 3901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.376866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.376897 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:34.377347 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.377366 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.379619 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.379624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.381945 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.381962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.383908 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.383915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.384121 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384194 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.384333 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.384535 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2716,"mutex_wait_us":28,"thread_start_us":3217,"threads_started":1}
I20260811 06:26:34.385274 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.370051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.7.4.129:45811)
0811 06:26:34.376182 (+ 6131us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.376192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.376225 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:26:34.376335 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.376342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.376597 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.377578 (+ 981us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:34.377812 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.379447 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.382168 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.383688 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.383830 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384050 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.384820 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.384963 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5967,"thread_start_us":5825,"threads_started":1}
I20260811 06:26:34.385285 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.375850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.7.4.131:44273)
0811 06:26:34.376600 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.376612 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.376649 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:26:34.376750 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.376759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.376973 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.378968 (+ 1995us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:34.379242 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.380614 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.382742 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.383973 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.384138 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384360 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.384750 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.384920 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":439,"threads_started":1}
I20260811 06:26:34.385507 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.370886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33883 (local address 127.0.0.1:60236)
0811 06:26:34.375767 (+ 4881us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.375785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.375897 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.377557 (+ 1660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.377562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.377587 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:34.378017 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.378030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.380261 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.380267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.382631 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.382646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.384550 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.384556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.384777 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384825 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.384927 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.385067 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4756,"mutex_wait_us":60,"thread_start_us":4585,"threads_started":1}
I20260811 06:26:34.386345 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.369962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:44273 (local address 127.0.0.1:40120)
0811 06:26:34.372001 (+ 2039us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.372051 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.372255 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.377205 (+ 4950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.377210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.377240 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:34.377681 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.377697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.380750 (+ 3053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.380758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.382540 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.382557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.384234 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.384242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.384436 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384471 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.384566 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.385971 (+ 1405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":1254,"thread_start_us":2619,"threads_started":1}
I20260811 06:26:34.386772 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.376392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.7.4.130:33883)
0811 06:26:34.376962 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.376970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.377005 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:26:34.377100 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.377109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.377333 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.378220 (+ 887us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:34.378516 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.380106 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.382817 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.384290 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.384464 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.384688 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.386287 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.386435 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":265,"threads_started":1}
I20260811 06:26:34.389032 23270 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 (DEFAULT_TABLE table=test-table [id=2701741f08e54276ac90de0e909cad14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:34.390257 23270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecf41abab1c48e08d7e0ce5696efac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:34.389947 23572 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 (DEFAULT_TABLE table=test-table [id=2701741f08e54276ac90de0e909cad14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:34.389947 23422 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 (DEFAULT_TABLE table=test-table [id=2701741f08e54276ac90de0e909cad14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:34.391228 23422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecf41abab1c48e08d7e0ce5696efac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:34.391232 23572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecf41abab1c48e08d7e0ce5696efac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:34.402540 23662 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Bootstrap starting.
I20260811 06:26:34.405613 23662 tablet_bootstrap.cc:654] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:34.406210 23663 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Bootstrap starting.
I20260811 06:26:34.407194 23665 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Bootstrap starting.
I20260811 06:26:34.407552 23662 log.cc:824] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:34.409945 23663 tablet_bootstrap.cc:654] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:34.410490 23665 tablet_bootstrap.cc:654] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:34.411159 23344 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:34.411211 23663 log.cc:824] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:34.411468 23662 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: No bootstrap required, opened a new log
I20260811 06:26:34.411669 23665 log.cc:824] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:34.411733 23662 ts_tablet_manager.cc:1397] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:26:34.420469 23663 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: No bootstrap required, opened a new log
I20260811 06:26:34.420734 23663 ts_tablet_manager.cc:1397] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260811 06:26:34.420780 23665 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: No bootstrap required, opened a new log
I20260811 06:26:34.421020 23665 ts_tablet_manager.cc:1397] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260811 06:26:34.423036 23662 raft_consensus.cc:348] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.423444 23662 raft_consensus.cc:374] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:34.423566 23662 raft_consensus.cc:729] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbf8dd867754106a65a9c6eb8d0d35c, State: Initialized, Role: FOLLOWER
I20260811 06:26:34.424154 23662 consensus_queue.cc:260] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.425909 23662 ts_tablet_manager.cc:1428] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 06:26:34.431478 23665 raft_consensus.cc:348] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.431478 23663 raft_consensus.cc:348] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.431900 23663 raft_consensus.cc:374] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:34.431900 23665 raft_consensus.cc:374] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:34.432005 23665 raft_consensus.cc:729] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c90a403c904d08999354cb946c1480, State: Initialized, Role: FOLLOWER
I20260811 06:26:34.432005 23663 raft_consensus.cc:729] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: debb097781464bb49be8bfa30f186a2e, State: Initialized, Role: FOLLOWER
I20260811 06:26:34.432605 23665 consensus_queue.cc:260] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.432605 23663 consensus_queue.cc:260] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.433696 23499 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:34.434172 23643 heartbeater.cc:503] Master 127.7.4.190:45995 was elected leader, sending a full tablet report...
I20260811 06:26:34.434461 23663 ts_tablet_manager.cc:1428] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260811 06:26:34.434810 23665 ts_tablet_manager.cc:1428] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260811 06:26:34.537143 23672 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:34.537441 23672 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.539280 23672 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811), f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:34.546483 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.539907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:57525 (local address 127.7.4.131:44273)
0811 06:26:34.540129 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.540138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.541033 (+ 895us) server_negotiation.cc:407] Connection header received
0811 06:26:34.541224 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.541227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.541322 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.541418 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:34.542041 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.542784 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.544668 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.545479 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.545578 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.545693 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.546070 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.546219 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:26:34.546504 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.539800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38949 (local address 127.7.4.129:45811)
0811 06:26:34.540036 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.540044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.540915 (+ 871us) server_negotiation.cc:407] Connection header received
0811 06:26:34.541108 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.541116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.541202 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.541287 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:34.542138 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.542862 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.544764 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.545559 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.545708 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.545816 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.546132 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.546277 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:26:34.546600 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.539792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:44273 (local address 127.7.4.130:57525)
0811 06:26:34.540890 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.540912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.541080 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.541469 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.541472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.541503 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:34.541860 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.541871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.542937 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.542945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.544496 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.544512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.545712 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.545719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.545898 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.545923 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.545999 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.546128 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":12,"thread_start_us":566,"threads_started":1}
I20260811 06:26:34.546828 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.539750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.7.4.130:38949)
0811 06:26:34.540785 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.540801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:34.540934 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:34.541333 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:34.541337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:34.541383 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:34.541979 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.541993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.542999 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.543003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.544558 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:34.544596 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.545805 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.545810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:34.545947 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.545970 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:26:34.546089 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:26:34.546210 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":65,"thread_start_us":677,"threads_started":1}
I20260811 06:26:34.547863 23593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecf41abab1c48e08d7e0ce5696efac0" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7c90a403c904d08999354cb946c1480" is_pre_election: true
I20260811 06:26:34.547865 23293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecf41abab1c48e08d7e0ce5696efac0" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" is_pre_election: true
I20260811 06:26:34.548365 23293 raft_consensus.cc:2454] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 0.
I20260811 06:26:34.548363 23593 raft_consensus.cc:2454] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 0.
I20260811 06:26:34.549023 23370 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: debb097781464bb49be8bfa30f186a2e, efbf8dd867754106a65a9c6eb8d0d35c; no voters:
I20260811 06:26:34.549425 23672 raft_consensus.cc:2781] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:26:34.549584 23672 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:26:34.549703 23672 raft_consensus.cc:3037] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:34.552053 23672 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.552659 23672 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 election: Requested vote from peers efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811), f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273)
I20260811 06:26:34.553160 23593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecf41abab1c48e08d7e0ce5696efac0" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7c90a403c904d08999354cb946c1480"
I20260811 06:26:34.553153 23293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecf41abab1c48e08d7e0ce5696efac0" candidate_uuid: "debb097781464bb49be8bfa30f186a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
I20260811 06:26:34.553361 23593 raft_consensus.cc:3037] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:34.553361 23293 raft_consensus.cc:3037] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:34.556643 23593 raft_consensus.cc:2454] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 1.
I20260811 06:26:34.556658 23293 raft_consensus.cc:2454] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate debb097781464bb49be8bfa30f186a2e in term 1.
I20260811 06:26:34.557086 23370 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: debb097781464bb49be8bfa30f186a2e, efbf8dd867754106a65a9c6eb8d0d35c; no voters:
I20260811 06:26:34.557396 23672 raft_consensus.cc:2781] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:34.557889 23672 raft_consensus.cc:686] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 1 LEADER]: Becoming Leader. State: Replica: debb097781464bb49be8bfa30f186a2e, State: Running, Role: LEADER
I20260811 06:26:34.558394 23672 consensus_queue.cc:237] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:34.562974 23133 catalog_manager.cc:5495] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e reported cstate change: term changed from 0 to 1, leader changed from <none> to debb097781464bb49be8bfa30f186a2e (127.7.4.130). New cstate: current_term: 1 leader_uuid: "debb097781464bb49be8bfa30f186a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } health_report { overall_health: HEALTHY } } }
W20260811 06:26:34.566303 23644 tablet.cc:2369] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:26:34.614836 23500 tablet.cc:2369] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:34.617444 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.611928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.7.4.130:33883)
0811 06:26:34.612143 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.612162 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.612183 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:26:34.612442 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.612446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.612538 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.612718 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:26:34.613099 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.613805 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.615338 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.615775 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.615912 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.616390 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:26:34.616845 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:26:34.617003 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.617102 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.617189 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 06:26:34.625154 23345 tablet.cc:2369] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:34.625415 23593 raft_consensus.cc:1261] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Refusing update from remote peer debb097781464bb49be8bfa30f186a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:34.625666 23293 raft_consensus.cc:1261] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Refusing update from remote peer debb097781464bb49be8bfa30f186a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:34.626425 23675 consensus_queue.cc:1035] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:34.626763 23672 consensus_queue.cc:1035] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:34.639659 23677 mvcc.cc:205] Tried to move back new op lower bound from 7317215619575275520 to 7317215619315240960. Current Snapshot: MvccSnapshot[applied={T|T < 7317215619575275520}]
I20260811 06:26:34.644271 23678 mvcc.cc:205] Tried to move back new op lower bound from 7317215619575275520 to 7317215619315240960. Current Snapshot: MvccSnapshot[applied={T|T < 7317215619575275520}]
I20260811 06:26:34.661330 23133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:34.661594 23133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:34.665234 23133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Sending DeleteTablet for 3 replicas of tablet 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:34.666474 23572 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:40120
I20260811 06:26:34.666548 23270 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:59168
I20260811 06:26:34.666559 23422 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecf41abab1c48e08d7e0ce5696efac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:34 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60236
I20260811 06:26:34.669925 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23179
I20260811 06:26:34.676512 23692 tablet_replica.cc:330] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: stopping tablet replica
I20260811 06:26:34.677052 23692 raft_consensus.cc:2229] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:34.677977 23692 raft_consensus.cc:2258] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:34.678911 23691 tablet_replica.cc:330] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: stopping tablet replica
I20260811 06:26:34.680907 23692 ts_tablet_manager.cc:1905] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:34.681447 23691 raft_consensus.cc:2229] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:34.682875 23693 tablet_replica.cc:330] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: stopping tablet replica
I20260811 06:26:34.683233 23693 raft_consensus.cc:2229] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:26:34.683997 23693 raft_consensus.cc:2258] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:34.685881 23693 ts_tablet_manager.cc:1905] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:34.688691 23692 ts_tablet_manager.cc:1918] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:34.689024 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.670979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.7.4.129:45811)
0811 06:26:34.671207 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.671216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.671244 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:26:34.671518 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.671528 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.671672 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.671816 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:34.674059 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.675649 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.677617 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.678533 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.678728 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.682479 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:34.682497 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:34.682517 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:34.682571 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:34.685816 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.686773 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.686786 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.686789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.686926 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.687395 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.687402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.687405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.687763 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:34.688064 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.688364 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.688600 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:26:34.689242 23691 raft_consensus.cc:2258] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:34.689785 23339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:34.690356 23692 log.cc:1196] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:34.691021 23692 ts_tablet_manager.cc:1939] T 4ecf41abab1c48e08d7e0ce5696efac0 P f7c90a403c904d08999354cb946c1480: Deleting consensus metadata
I20260811 06:26:34.692981 23118 catalog_manager.cc:4841] TS f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): tablet 4ecf41abab1c48e08d7e0ce5696efac0 (table test-table [id=2701741f08e54276ac90de0e909cad14]) successfully deleted
I20260811 06:26:34.693419 23693 ts_tablet_manager.cc:1918] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:34.693641 23693 log.cc:1196] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:34.694209 23693 ts_tablet_manager.cc:1939] T 4ecf41abab1c48e08d7e0ce5696efac0 P debb097781464bb49be8bfa30f186a2e: Deleting consensus metadata
I20260811 06:26:34.695621 23119 catalog_manager.cc:4841] TS debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): tablet 4ecf41abab1c48e08d7e0ce5696efac0 (table test-table [id=2701741f08e54276ac90de0e909cad14]) successfully deleted
I20260811 06:26:34.786372 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23179
I20260811 06:26:34.816075 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23348
W20260811 06:26:34.816159 23120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.4.129:45811 (error 108)
I20260811 06:26:34.818595 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.818070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59190)
0811 06:26:34.818281 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.818416 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 06:26:34.818861 23120 catalog_manager.cc:4584] TS efbf8dd867754106a65a9c6eb8d0d35c (127.7.4.129:45811): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ecf41abab1c48e08d7e0ce5696efac0: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
I20260811 06:26:34.835420 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.816934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.7.4.130:33883)
0811 06:26:34.817128 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.817135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.817440 (+ 305us) server_negotiation.cc:407] Connection header received
0811 06:26:34.817519 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.817524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.817615 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.817736 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:34.819474 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.820276 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.821640 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.822084 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.822238 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.829100 (+ 6862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:34.829113 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:34.829128 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:34.829172 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:34.832459 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.833161 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.833170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.833174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.833340 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.833729 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.833735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.833739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.834104 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:34.834505 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.834789 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.835016 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:26:34.836122 23493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:34.872273 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.871763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59204)
0811 06:26:34.871976 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.872087 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:26:34.873332 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.872924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59212)
0811 06:26:34.873081 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.873188 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:26:34.933840 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.933319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59226)
0811 06:26:34.933544 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.933653 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:26:34.934930 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.934543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59236)
0811 06:26:34.934701 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:34.934783 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:26:34.951363 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23348
I20260811 06:26:34.976702 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23502
I20260811 06:26:34.989594 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:34.977412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.7.4.131:44273)
0811 06:26:34.977574 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:34.977579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:34.977601 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:26:34.977771 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:34.977775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:34.977842 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:34.977926 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:34.979248 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.980006 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.981316 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:34.981685 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:34.981779 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:34.984431 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:34.984444 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:34.984459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:34.984491 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:34.987034 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.987785 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.987792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.987795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.987931 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:34.988288 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:34.988291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:34.988293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:34.988648 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:34.988870 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:34.989128 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 06:26:34.989296 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:26:34.990367 23636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:35.015408 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:35.014842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59250)
0811 06:26:35.015069 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:35.015174 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:26:35.016474 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:35.016057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59256)
0811 06:26:35.016261 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:35.016344 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 06:26:35.082151 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23502
I20260811 06:26:35.103188 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23098
I20260811 06:26:35.104431 23171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:35.220781 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:35.220119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45811 (local address 127.0.0.1:59260)
0811 06:26:35.220305 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:35.220439 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 06:26:35.221318 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23098
I20260811 06:26:35.249043 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--webserver_interface=127.7.4.190
--webserver_port=32825
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:35.586169 23699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:35.586494 23699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:35.586666 23699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:35.594340 23699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:35.594566 23699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:35.594696 23699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:35.594815 23699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:35.594900 23699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:35.604996 23699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:35.606354 23699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:35.607676 23699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:35.619455 23706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:35.619572 23705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:35.619676 23708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:35.620311 23699 server_base.cc:1034] running on GCE node
I20260811 06:26:35.621035 23699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:35.622351 23699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:35.623574 23699 hybrid_clock.cc:648] HybridClock initialized: now 1786429595623532 us; error 55 us; skew 500 ppm
I20260811 06:26:35.623911 23699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:35 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429595, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:35.627139 23699 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:35.628526 23699 webserver.cc:460] Webserver started at http://127.7.4.190:32825/ using document root <none> and password file <none>
I20260811 06:26:35.629195 23699 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:35.629331 23699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:35.634585 23699 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 06:26:35.637291 23715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:35.638487 23699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:26:35.638661 23699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "2d3b72ecb85b4b278ef4982c26b3a61f"
format_stamp: "Formatted at 2026-08-11 06:26:29 on dist-test-slave-07c2"
server_key: "d365a0950ca8251e8265f12d4fd9674e"
server_key_iv: "43952e37f8f74ca4f8882bb967f4b7d3"
server_key_version: "encryptionkey@0"
I20260811 06:26:35.639705 23699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:35.652138 23699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:35.655779 23699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:35.656054 23699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:35.673884 23699 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45995
I20260811 06:26:35.673918 23776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45995 every 8 connection(s)
I20260811 06:26:35.675334 23699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:35.676272 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23699
I20260811 06:26:35.677182 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:45811
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=33687
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:35.682276 23777 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:26:35.685499 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap starting.
I20260811 06:26:35.713380 23777 log.cc:824] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:35.734440 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:26:35.735086 23777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f: Bootstrap complete.
I20260811 06:26:35.749168 23777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:35.750418 23777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Initialized, Role: FOLLOWER
I20260811 06:26:35.751228 23777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:35.751564 23777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:35.751705 23777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:35.751925 23777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:26:35.758281 23777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:35.758844 23777 leader_election.cc:302] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d3b72ecb85b4b278ef4982c26b3a61f; no voters:
I20260811 06:26:35.759544 23777 leader_election.cc:288] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:26:35.759685 23782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:26:35.760998 23782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [term 3 LEADER]: Becoming Leader. State: Replica: 2d3b72ecb85b4b278ef4982c26b3a61f, State: Running, Role: LEADER
I20260811 06:26:35.761591 23777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:35.761513 23782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } }
I20260811 06:26:35.764083 23785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d3b72ecb85b4b278ef4982c26b3a61f. Latest consensus state: current_term: 3 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:35.764484 23785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:35.764374 23784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3b72ecb85b4b278ef4982c26b3a61f" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45995 } } }
I20260811 06:26:35.764644 23784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3b72ecb85b4b278ef4982c26b3a61f [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:35.769122 23794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:35.778187 23794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:35.779650 23794 catalog_manager.cc:1259] Loaded cluster ID: 9e5b744717b544499fed90394b9b41f2
I20260811 06:26:35.779747 23794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 06:26:35.781963 23794 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c1c7059ad google::LogMessage::Fail() at ??:0
@ 0x7f2c1c70888a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c1c7054de google::LogMessage::Flush() at ??:0
@ 0x7f2c1c706a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c3213c512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2c32139976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c321c1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2c321c1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2c1f351345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c1f35491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2c1f354799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2c1f32feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c226096db start_thread at ??:0
@ 0x7f2c1b59f61f clone at ??:0
W20260811 06:26:36.073604 23779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:36.073911 23779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:36.074083 23779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:36.081728 23779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:36.081868 23779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:36.082049 23779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:36.091629 23779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:45811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=33687
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:36.092811 23779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:36.094087 23779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:36.106973 23829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:36.107173 23828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:36.107180 23831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:36.108559 23779 server_base.cc:1034] running on GCE node
I20260811 06:26:36.109175 23779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:36.110105 23779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:36.111372 23779 hybrid_clock.cc:648] HybridClock initialized: now 1786429596111315 us; error 45 us; skew 500 ppm
I20260811 06:26:36.111675 23779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:36 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429596, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:36.114640 23779 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:36.116072 23779 webserver.cc:460] Webserver started at http://127.7.4.129:33687/ using document root <none> and password file <none>
I20260811 06:26:36.116705 23779 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:36.116847 23779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:36.122184 23779 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 06:26:36.127223 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:36.128453 23779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 06:26:36.128681 23779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "efbf8dd867754106a65a9c6eb8d0d35c"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "3c4fc0a22a7726f9072f12c639db1e50"
server_key_iv: "ee8015bc86ce3f252ba246109f51731c"
server_key_version: "encryptionkey@0"
I20260811 06:26:36.129618 23779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:36.152237 23779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:36.155907 23779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:36.156178 23779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:36.157128 23779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:36.158942 23848 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 06:26:36.164335 23779 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 06:26:36.164494 23779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 06:26:36.164641 23779 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 06:26:36.167673 23779 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 06:26:36.167783 23779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:36.168088 23848 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Bootstrap starting.
I20260811 06:26:36.176133 23848 log.cc:824] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:36.189851 23848 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:26:36.190793 23848 tablet_bootstrap.cc:492] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Bootstrap complete.
I20260811 06:26:36.191244 23848 ts_tablet_manager.cc:1397] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.005s
I20260811 06:26:36.200575 23848 raft_consensus.cc:348] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:36.202150 23848 raft_consensus.cc:729] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbf8dd867754106a65a9c6eb8d0d35c, State: Initialized, Role: FOLLOWER
I20260811 06:26:36.202832 23848 consensus_queue.cc:260] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:36.204344 23848 ts_tablet_manager.cc:1428] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
I20260811 06:26:36.214254 23779 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:45811
I20260811 06:26:36.214313 23970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:45811 every 8 connection(s)
I20260811 06:26:36.215513 23779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:36.219908 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.218035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:36927)
0811 06:26:36.218816 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.219003 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":355,"threads_started":1}
W20260811 06:26:36.220558 23971 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45995 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45995: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
I20260811 06:26:36.221339 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.221032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:52303)
0811 06:26:36.221168 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.221232 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 06:26:36.222259 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.221955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:37589)
0811 06:26:36.222077 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.222151 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:26:36.222518 23971 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:26:36.223277 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23779
I20260811 06:26:36.223940 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:33883
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=38777
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:36.555018 23974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:36.555361 23974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:36.555537 23974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:36.562855 23974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:36.562949 23974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:36.563054 23974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:36.573429 23974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:33883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=38777
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:36.574657 23974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:36.576017 23974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:36.588846 23980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:36.588913 23981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:36.588943 23983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:36.589919 23974 server_base.cc:1034] running on GCE node
I20260811 06:26:36.590514 23974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:36.591467 23974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:36.592694 23974 hybrid_clock.cc:648] HybridClock initialized: now 1786429596592655 us; error 46 us; skew 500 ppm
I20260811 06:26:36.592998 23974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:36 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429596, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:36.596148 23974 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:36.597477 23974 webserver.cc:460] Webserver started at http://127.7.4.130:38777/ using document root <none> and password file <none>
I20260811 06:26:36.598165 23974 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:36.598309 23974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:36.603523 23974 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:26:36.606036 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:36.607366 23974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:36.607532 23974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "debb097781464bb49be8bfa30f186a2e"
format_stamp: "Formatted at 2026-08-11 06:26:30 on dist-test-slave-07c2"
server_key: "49bf5ce1b9e973faae33cce8e010a02a"
server_key_iv: "e5bbd7b5504e1d46fc057bbc1e888467"
server_key_version: "encryptionkey@0"
I20260811 06:26:36.608469 23974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:36.628341 23974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:36.632493 23974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:36.632774 23974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:36.633682 23974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:36.634976 23974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:36.635111 23974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:36.635247 23974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:36.635353 23974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 06:26:36.669054 23974 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:33883
I20260811 06:26:36.669135 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:33883 every 8 connection(s)
I20260811 06:26:36.670420 23974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:36.673273 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 23974
I20260811 06:26:36.674094 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:44273
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:45995
--webserver_port=37695
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:26:36.675414 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.673440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:45669)
0811 06:26:36.674299 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.674532 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":350,"threads_started":1}
W20260811 06:26:36.676072 24129 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45995 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45995: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
I20260811 06:26:36.676925 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.676608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:44071)
0811 06:26:36.676751 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.676829 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:26:36.677807 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:36.677435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:36179)
0811 06:26:36.677574 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:36.677649 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:26:36.678159 24129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:26:37.007400 24132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:37.007699 24132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:37.007889 24132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:37.015197 24132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:37.015341 24132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:37.015522 24132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:37.024840 24132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:44273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=37695
--tserver_master_addrs=127.7.4.190:45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:37.026036 24132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:37.027396 24132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:37.040396 24139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:37.040488 24138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:37.040517 24141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:37.041419 24132 server_base.cc:1034] running on GCE node
I20260811 06:26:37.042012 24132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:37.042941 24132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:37.044163 24132 hybrid_clock.cc:648] HybridClock initialized: now 1786429597044140 us; error 62 us; skew 500 ppm
I20260811 06:26:37.044482 24132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:37 dist-test-slave-07c2 krb5kdc[22463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:37.047906 24132 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:37.049360 24132 webserver.cc:460] Webserver started at http://127.7.4.131:37695/ using document root <none> and password file <none>
I20260811 06:26:37.049989 24132 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:37.050094 24132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:37.055262 24132 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:26:37.058208 24148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:37.059566 24132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:37.059700 24132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "f7c90a403c904d08999354cb946c1480"
format_stamp: "Formatted at 2026-08-11 06:26:31 on dist-test-slave-07c2"
server_key: "16826d7e81cf8c095266b6ddf1cfc0dd"
server_key_iv: "c7b23283ee391c5beb17ae04d2971119"
server_key_version: "encryptionkey@0"
I20260811 06:26:37.060616 24132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:37.090193 24132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:37.094206 24132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:37.094458 24132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:37.095458 24132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:37.096735 24132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:37.096822 24132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:37.096906 24132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:37.096962 24132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:37.134354 24132 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:44273
I20260811 06:26:37.134409 24272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:44273 every 8 connection(s)
I20260811 06:26:37.135953 24132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:37.141054 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.138940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:60697)
0811 06:26:37.139889 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.140210 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":374,"threads_started":1}
W20260811 06:26:37.141600 24273 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45995 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45995: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
I20260811 06:26:37.142380 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.142084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:59637)
0811 06:26:37.142222 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.142284 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:26:37.143244 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.142872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:58651)
0811 06:26:37.142986 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.143087 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:26:37.143474 24273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:26:37.143505 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24132
I20260811 06:26:37.223805 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.223003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:60219)
0811 06:26:37.223501 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.223581 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":204,"threads_started":1}
I20260811 06:26:37.570431 24281 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:37.570739 24281 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:37.572886 24281 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273), debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
Aug 11 06:26:37 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429596, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
Aug 11 06:26:37 dist-test-slave-07c2 krb5kdc[22463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429596, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:26:37.595873 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.574023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:33883 (local address 127.7.4.129:33481)
0811 06:26:37.576132 (+ 2109us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.576154 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:37.576301 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:37.579326 (+ 3025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:37.579330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:37.580005 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:37.580917 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:37.580946 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.583139 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.583144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:37.584296 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:37.584309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.585303 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.585310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:37.585618 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:37.586368 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:37.586399 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:37.589307 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:37.592551 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:37.592557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:37.592561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:37.593034 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:37.593478 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:37.593487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:37.593676 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:37.594679 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:37.594683 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:37.594921 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 06:26:37.595142 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 06:26:37.595548 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1959,"mutex_wait_us":59,"spinlock_wait_cycles":24960,"thread_start_us":1789,"threads_started":1}
I20260811 06:26:37.596167 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.573232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:44273 (local address 127.7.4.129:39493)
0811 06:26:37.573846 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.573897 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:37.574489 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:37.578788 (+ 4299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:37.578799 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:37.579810 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:37.580919 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:37.580948 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.582487 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.582492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:37.584124 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:37.584137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.585257 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.585264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:37.585617 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:37.586440 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:37.586457 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:37.588969 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:37.592487 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:37.592496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:37.592509 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:37.593016 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:37.593450 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:37.593459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:37.593646 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:37.594651 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:37.594658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:37.594922 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 06:26:37.595162 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 06:26:37.595733 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":299,"spinlock_wait_cycles":3968}
I20260811 06:26:37.596242 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.573544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:39493 (local address 127.7.4.131:44273)
0811 06:26:37.574334 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:37.574382 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:37.574476 (+ 94us) server_negotiation.cc:407] Connection header received
0811 06:26:37.577677 (+ 3201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:37.577690 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:37.578174 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:37.578535 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:37.581117 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.582305 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.584267 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.585046 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.585251 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:37.589257 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:37.589264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:37.589276 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:37.589317 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:37.592342 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593152 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:37.593166 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:37.593171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:37.593333 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593771 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:37.593779 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:37.593781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:37.594403 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:37.594604 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:37.595435 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 06:26:37.595831 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":1420,"threads_started":1}
I20260811 06:26:37.597465 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.573601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:33481 (local address 127.7.4.130:33883)
0811 06:26:37.575636 (+ 2035us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:37.575652 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:37.576273 (+ 621us) server_negotiation.cc:407] Connection header received
0811 06:26:37.578589 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:37.578599 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:37.579004 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:37.579358 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:37.581169 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.582959 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.584444 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:37.585096 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:37.585290 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:37.589574 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:37.589581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:37.589592 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:37.589625 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:37.592408 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593179 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:37.593185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:37.593191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:37.593358 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:37.593829 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:37.593836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:37.593841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:37.594422 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:37.594606 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:37.596677 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0811 06:26:37.597060 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1836,"thread_start_us":1690,"threads_started":1}
W20260811 06:26:37.599566 23840 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
W20260811 06:26:37.599956 23842 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:37.600096 23842 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efbf8dd867754106a65a9c6eb8d0d35c; no voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480
I20260811 06:26:37.600499 24281 raft_consensus.cc:2726] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:26:37.679595 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:37.678613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:52613)
0811 06:26:37.679145 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:37.679352 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":285,"threads_started":1}
I20260811 06:26:38.144624 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:38.143920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:57293)
0811 06:26:38.144338 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:38.144436 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":206,"threads_started":1}
I20260811 06:26:38.225251 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:38.224635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:40051)
0811 06:26:38.225033 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:38.225107 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":207,"threads_started":1}
I20260811 06:26:38.681015 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:38.680372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:49263)
0811 06:26:38.680791 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:38.680867 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":218,"threads_started":1}
I20260811 06:26:39.146212 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:39.145544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:39337)
0811 06:26:39.145951 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:39.146034 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":222,"threads_started":1}
I20260811 06:26:39.227242 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:39.226403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:49215)
0811 06:26:39.226931 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:39.227042 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":289,"threads_started":1}
I20260811 06:26:39.682690 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:39.681945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:53751)
0811 06:26:39.682429 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:39.682515 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":242,"threads_started":1}
I20260811 06:26:39.756506 24292 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:39.756733 24292 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:39.757364 24292 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273), debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
W20260811 06:26:39.758550 23842 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
W20260811 06:26:39.758749 23840 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:39.758859 23840 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efbf8dd867754106a65a9c6eb8d0d35c; no voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480
I20260811 06:26:39.759104 24292 raft_consensus.cc:2726] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:26:40.147766 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:40.147038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:52049)
0811 06:26:40.147536 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:40.147610 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":318,"threads_started":1}
I20260811 06:26:40.228842 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:40.228085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:49999)
0811 06:26:40.228571 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:40.228660 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":256,"threads_started":1}
I20260811 06:26:40.684338 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:40.683631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:45879)
0811 06:26:40.684074 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:40.684160 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":229,"threads_started":1}
I20260811 06:26:41.149363 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:41.148654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:48107)
0811 06:26:41.149104 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:41.149178 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":225,"threads_started":1}
I20260811 06:26:41.230549 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:41.229799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:40887)
0811 06:26:41.230291 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:41.230368 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":221,"threads_started":1}
I20260811 06:26:41.685878 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:41.685204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:53221)
0811 06:26:41.685642 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:41.685724 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":238,"threads_started":1}
I20260811 06:26:42.150913 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:42.150209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:49325)
0811 06:26:42.150679 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:42.150763 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":204,"threads_started":1}
I20260811 06:26:42.232157 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:42.231481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:49305)
0811 06:26:42.231932 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:42.232005 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":256,"threads_started":1}
I20260811 06:26:42.306366 24304 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:42.306541 24304 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:42.307137 24304 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273), debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
W20260811 06:26:42.308159 23840 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
W20260811 06:26:42.308374 23842 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:42.308538 23842 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efbf8dd867754106a65a9c6eb8d0d35c; no voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480
I20260811 06:26:42.308826 24304 raft_consensus.cc:2726] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:26:42.687672 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:42.686894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:56045)
0811 06:26:42.687406 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:42.687482 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":267,"threads_started":1}
I20260811 06:26:43.152427 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:43.151747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:42459)
0811 06:26:43.152178 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:43.152261 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":232,"threads_started":1}
I20260811 06:26:43.233744 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:43.233043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:34351)
0811 06:26:43.233487 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:43.233562 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":260,"threads_started":1}
I20260811 06:26:43.689260 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:43.688496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:46029)
0811 06:26:43.689002 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:43.689088 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":224,"threads_started":1}
I20260811 06:26:44.154063 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:44.153273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:48589)
0811 06:26:44.153790 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:44.153866 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":223,"threads_started":1}
I20260811 06:26:44.235443 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:44.234599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:56315)
0811 06:26:44.235140 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:44.235229 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":328,"threads_started":1}
I20260811 06:26:44.264221 24311 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:44.264380 24311 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:44.265079 24311 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273), debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
W20260811 06:26:44.266026 23840 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
W20260811 06:26:44.266220 23842 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:44.266357 23842 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efbf8dd867754106a65a9c6eb8d0d35c; no voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480
I20260811 06:26:44.266573 24311 raft_consensus.cc:2726] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:26:44.690889 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:44.690162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:57411)
0811 06:26:44.690603 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:44.690685 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260811 06:26:45.155601 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:45.154945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:38811)
0811 06:26:45.155358 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:45.155434 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":203,"threads_started":1}
I20260811 06:26:45.236891 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:45.236215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:49257)
0811 06:26:45.236641 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:45.236725 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260811 06:26:45.692499 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:45.691822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:52419)
0811 06:26:45.692267 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:45.692350 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":216,"threads_started":1}
I20260811 06:26:46.157199 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:46.156467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:48047)
0811 06:26:46.156943 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:46.157031 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260811 06:26:46.238667 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:46.237980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:35907)
0811 06:26:46.238377 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:46.238461 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":213,"threads_started":1}
I20260811 06:26:46.539426 24319 raft_consensus.cc:482] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:46.539628 24319 raft_consensus.cc:504] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efbf8dd867754106a65a9c6eb8d0d35c" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45811 } } peers { permanent_uuid: "f7c90a403c904d08999354cb946c1480" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 44273 } } peers { permanent_uuid: "debb097781464bb49be8bfa30f186a2e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 33883 } }
I20260811 06:26:46.540306 24319 leader_election.cc:288] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273), debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883)
W20260811 06:26:46.541214 23840 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7c90a403c904d08999354cb946c1480 (127.7.4.131:44273): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
W20260811 06:26:46.541407 23842 leader_election.cc:341] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer debb097781464bb49be8bfa30f186a2e (127.7.4.130:33883): Not found: Tablet not found: 4ecf41abab1c48e08d7e0ce5696efac0
I20260811 06:26:46.541575 23842 leader_election.cc:302] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efbf8dd867754106a65a9c6eb8d0d35c; no voters: debb097781464bb49be8bfa30f186a2e, f7c90a403c904d08999354cb946c1480
I20260811 06:26:46.541826 24319 raft_consensus.cc:2726] T 4ecf41abab1c48e08d7e0ce5696efac0 P efbf8dd867754106a65a9c6eb8d0d35c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:26:46.694082 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:46.693441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:34135)
0811 06:26:46.693842 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:46.693916 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":218,"threads_started":1}
I20260811 06:26:47.158800 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:47.158137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:41447)
0811 06:26:47.158541 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:47.158617 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":212,"threads_started":1}
I20260811 06:26:47.240511 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:47.239733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:44961)
0811 06:26:47.240233 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:47.240319 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":250,"threads_started":1}
I20260811 06:26:47.695747 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:47.694997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:54291)
0811 06:26:47.695501 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:47.695590 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":216,"threads_started":1}
I20260811 06:26:48.160501 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:48.159810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:46685)
0811 06:26:48.160257 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:48.160345 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":225,"threads_started":1}
I20260811 06:26:48.242084 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:48.241398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:51031)
0811 06:26:48.241828 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:48.241914 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":234,"threads_started":1}
I20260811 06:26:48.697234 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:48.696571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:46881)
0811 06:26:48.696993 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:48.697067 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":232,"threads_started":1}
I20260811 06:26:49.162199 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:49.161415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:42239)
0811 06:26:49.161834 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:49.161959 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":242,"threads_started":1}
I20260811 06:26:49.243770 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:49.242983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:35299)
0811 06:26:49.243504 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:49.243583 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":255,"threads_started":1}
I20260811 06:26:49.698781 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:49.698083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:33611)
0811 06:26:49.698523 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:49.698599 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":234,"threads_started":1}
I20260811 06:26:50.163877 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:50.163088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:53957)
0811 06:26:50.163616 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:50.163692 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":309,"threads_started":1}
I20260811 06:26:50.245290 24331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:50.244584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:37681)
0811 06:26:50.245005 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:50.245097 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":216,"threads_started":1}
I20260811 06:26:50.700443 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:50.699702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:47513)
0811 06:26:50.700171 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:50.700248 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":233,"threads_started":1}
I20260811 06:26:51.165560 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:51.164730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:34301)
0811 06:26:51.165295 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:51.165370 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":251,"threads_started":1}
I20260811 06:26:51.247165 24335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:51.246448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:49147)
0811 06:26:51.246923 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:51.246999 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":223,"threads_started":1}
I20260811 06:26:51.702309 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:51.701658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.130:58661)
0811 06:26:51.702067 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:51.702158 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260811 06:26:52.147913 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23779
I20260811 06:26:52.166544 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:52.149142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.7.4.129:45811)
0811 06:26:52.149831 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:52.149843 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:52.149880 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:26:52.149986 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:52.149997 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:52.150302 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:52.150487 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:52.152176 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.153696 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.155733 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.156731 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.156867 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:52.160503 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:52.160514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:52.160529 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:52.160560 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:52.163840 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.164549 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.164556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.164558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.164707 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.165031 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.165034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.165036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.165462 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:52.165714 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:52.165967 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:26:52.166234 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":340,"threads_started":1}
I20260811 06:26:52.167622 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:52.166593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.131:54601)
0811 06:26:52.167358 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:52.167436 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":497,"threads_started":1}
I20260811 06:26:52.167964 23966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:52.260392 24340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:52.257608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45995 (local address 127.7.4.129:52593)
0811 06:26:52.259890 (+ 2282us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:52.260068 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2116,"thread_start_us":1906,"threads_started":1}
I20260811 06:26:52.260919 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23779
I20260811 06:26:52.286464 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 23974
I20260811 06:26:52.299784 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:52.287214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.7.4.130:33883)
0811 06:26:52.287726 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:52.287730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:52.287754 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:26:52.287811 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:52.287815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:52.287874 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:52.287964 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:52.289400 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.290223 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.291485 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.291908 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.292002 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:52.294519 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:52.294523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:52.294526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:52.294556 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:52.297368 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.298020 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.298024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.298026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.298156 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.298459 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.298462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.298464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.298703 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:52.298908 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:52.299121 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 06:26:52.299279 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":269,"threads_started":1}
I20260811 06:26:52.300464 24124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:52.384703 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 23974
I20260811 06:26:52.406247 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24132
I20260811 06:26:52.419879 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:52.407174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.7.4.131:44273)
0811 06:26:52.407894 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:52.407899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:52.407917 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:26:52.407972 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:52.407976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:52.408033 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:52.408122 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:52.409521 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.410329 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.411657 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:52.412084 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:52.412177 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:52.414648 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:52.414653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:52.414656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:52.414688 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:52.417509 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.418095 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.418099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.418101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.418235 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:52.418555 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:52.418558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:52.418560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:52.418785 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:52.418996 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:52.419223 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 06:26:52.419462 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":450,"threads_started":1}
I20260811 06:26:52.420728 24267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:52.504022 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24132
2026-08-11T06:26:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24629 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:26:52 dist-test-slave-07c2 krb5kdc[24349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:26:52 dist-test-slave-07c2 krb5kdc[24349](info): set up 2 sockets
Aug 11 06:26:52 dist-test-slave-07c2 krb5kdc[24349](info): commencing operation
krb5kdc: starting...
W20260811 06:26:53.690488 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:26:53 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:26:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:26:53.968833 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:54.299286 24363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:54.299557 24363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:54.299645 24363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:54.306427 24363 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:26:54.306531 24363 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:26:54.306568 24363 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:26:54.306613 24363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:26:54.306658 24363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:54.306717 24363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:54.306762 24363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:26:54.306808 24363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:26:54.316236 24363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:54.317322 24363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:54.318531 24363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:54.330186 24368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:54.330155 24369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:54.330312 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:54.331619 24363 server_base.cc:1034] running on GCE node
I20260811 06:26:54.332373 24363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:54.333633 24363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:54.334875 24363 hybrid_clock.cc:648] HybridClock initialized: now 1786429614334820 us; error 31 us; skew 500 ppm
I20260811 06:26:54.335194 24363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:54 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:54.338181 24363 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:26:54.339601 24363 webserver.cc:460] Webserver started at http://127.7.4.190:40765/ using document root <none> and password file <none>
I20260811 06:26:54.340256 24363 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:54.340389 24363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:54.340708 24363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:54.343662 24363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "5478333bcb9c48ec8bee9d121d5ae088"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "492a719dcf839c4507385287bb47dc9e"
server_key_iv: "4289f58ebeabf0007e5a114d049436b0"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.344427 24363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "5478333bcb9c48ec8bee9d121d5ae088"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "492a719dcf839c4507385287bb47dc9e"
server_key_iv: "4289f58ebeabf0007e5a114d049436b0"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.349395 24363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:26:54.352766 24382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:54.354020 24363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:26:54.354200 24363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "5478333bcb9c48ec8bee9d121d5ae088"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "492a719dcf839c4507385287bb47dc9e"
server_key_iv: "4289f58ebeabf0007e5a114d049436b0"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.354421 24363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:54.369065 24363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:54.372838 24363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:54.373092 24363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:54.391862 24363 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45487
I20260811 06:26:54.391881 24447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45487 every 8 connection(s)
I20260811 06:26:54.393152 24363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:26:54.396667 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:54.402745 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24363
I20260811 06:26:54.402981 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:26:54.403869 7186 external_mini_cluster.cc:1428] Setting key 63005bb7e5a9b66f2d1278ad916df6b4
I20260811 06:26:54.408028 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap starting.
I20260811 06:26:54.411082 24448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:54.412164 24448 log.cc:824] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:54.414775 24448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: No bootstrap required, opened a new log
Aug 11 06:26:54 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:54.422851 24448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:26:54.423204 24448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:54.423360 24448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Initialized, Role: FOLLOWER
I20260811 06:26:54.423945 24448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:26:54.424214 24448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:26:54.424371 24448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:26:54.424542 24448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:54.426227 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:54.406046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.7.4.190:45487)
0811 06:26:54.406782 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:54.406793 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:54.406829 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:26:54.408689 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:54.408710 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:54.409053 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:54.409431 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:54.410952 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:54.412420 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:54.413787 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:54.414505 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:54.414718 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:54.419032 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:54.419039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:54.419052 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:54.419096 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:54.422247 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:54.422953 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:54.422967 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:54.422972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:54.423165 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:54.423607 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:54.423613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:54.423617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:54.424104 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:54.424436 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:54.424879 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 06:26:54.425222 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":328,"threads_started":1}
I20260811 06:26:54.427049 24448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:26:54.427579 24448 leader_election.cc:302] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5478333bcb9c48ec8bee9d121d5ae088; no voters:
I20260811 06:26:54.428201 24448 leader_election.cc:288] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:26:54.428354 24454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:54.429512 24454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 LEADER]: Becoming Leader. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Running, Role: LEADER
I20260811 06:26:54.429987 24454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:26:54.430320 24448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:26:54.432435 24456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5478333bcb9c48ec8bee9d121d5ae088. Latest consensus state: current_term: 1 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:26:54.432770 24456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:54.433386 24455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:26:54.433694 24459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:26:54.433763 24455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:26:54.440264 24459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:26:54.449753 24459 catalog_manager.cc:1347] Generated new cluster ID: b4999faa55924498a86ac2789ba0b8b1
I20260811 06:26:54.449852 24459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:26:54.469283 24459 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:26:54.471016 24459 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:26:54.479813 24459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Generated new TSK 0
I20260811 06:26:54.480772 24459 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:26:54.550444 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:54.885059 24475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:54.885330 24475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:54.885456 24475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:54.892576 24475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:54.892695 24475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:54.892875 24475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:26:54.902467 24475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:54.903856 24475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:54.905201 24475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:54.918875 24482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:54.918933 24485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:54.918910 24483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:54.919724 24475 server_base.cc:1034] running on GCE node
I20260811 06:26:54.920338 24475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:54.921288 24475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:54.922533 24475 hybrid_clock.cc:648] HybridClock initialized: now 1786429614922486 us; error 31 us; skew 500 ppm
I20260811 06:26:54.922848 24475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:54 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:54.926280 24475 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:26:54.928002 24475 webserver.cc:460] Webserver started at http://127.7.4.129:34863/ using document root <none> and password file <none>
I20260811 06:26:54.928763 24475 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:54.928915 24475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:54.929244 24475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:54.932564 24475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "8931e35c759180f6b50feb464e93c057"
server_key_iv: "a10f065c9255274d828f05e9c8127fcf"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.933365 24475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "8931e35c759180f6b50feb464e93c057"
server_key_iv: "a10f065c9255274d828f05e9c8127fcf"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.939076 24475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:26:54.942701 24492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:54.944201 24475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:26:54.944391 24475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "8931e35c759180f6b50feb464e93c057"
server_key_iv: "a10f065c9255274d828f05e9c8127fcf"
server_key_version: "encryptionkey@0"
I20260811 06:26:54.944604 24475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:54.962980 24475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:54.966701 24475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:54.966928 24475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:54.967849 24475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:54.969161 24475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:54.969246 24475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:54.969331 24475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:54.969375 24475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.002954 24475 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39133
I20260811 06:26:55.002997 24621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39133 every 8 connection(s)
I20260811 06:26:55.004201 24475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:26:55.009606 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24475
I20260811 06:26:55.009851 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:26:55.010641 7186 external_mini_cluster.cc:1428] Setting key a31bc9765fbbaadc9f25c16c64b9ea7d
Aug 11 06:26:55 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:55.024101 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:55.006607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:41089 (local address 127.7.4.190:45487)
0811 06:26:55.007025 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:55.007029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:55.008046 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 06:26:55.010022 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:55.010029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:55.010123 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:55.010257 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:55.011828 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.012604 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.014154 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.014572 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.014712 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:55.018395 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:55.018400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:55.018403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:55.018435 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:55.021045 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:55.021773 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:55.021779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:55.021781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:55.021901 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:55.022355 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:55.022359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:55.022361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:55.022613 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:55.022801 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:55.023503 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 06:26:55.023726 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":207,"threads_started":1}
I20260811 06:26:55.024999 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:55.007132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:41089)
0811 06:26:55.007893 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:55.007930 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:55.009702 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:55.010454 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:55.010461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:55.011035 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:55.011647 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:55.011666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.012784 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.012789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:55.013997 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:55.014010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.014826 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.014834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:55.015055 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:55.015772 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:55.015800 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:55.018159 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:55.021196 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:55.021205 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:55.021219 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:55.021631 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:55.022017 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:55.022022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:55.022025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:55.022217 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:55.022767 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:55.022775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:55.023098 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 06:26:55.023390 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 06:26:55.023894 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":335,"threads_started":1}
I20260811 06:26:55.026587 24622 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:26:55.026894 24622 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:55.027667 24622 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:26:55.029698 24405 ts_manager.cc:194] Registered new tserver with Master: 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133)
I20260811 06:26:55.031196 24405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:41089
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:26:55.077880 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:55.411310 24628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:55.411579 24628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:55.411676 24628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:55.418867 24628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:55.418991 24628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:55.419114 24628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:26:55.428298 24628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:55.429394 24628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:55.430723 24628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:55.443614 24634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:55.443555 24635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:55.444139 24637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:55.444998 24628 server_base.cc:1034] running on GCE node
I20260811 06:26:55.445526 24628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:55.446348 24628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:55.447577 24628 hybrid_clock.cc:648] HybridClock initialized: now 1786429615447493 us; error 71 us; skew 500 ppm
I20260811 06:26:55.447909 24628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:55 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:55.451079 24628 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:26:55.452579 24628 webserver.cc:460] Webserver started at http://127.7.4.130:43183/ using document root <none> and password file <none>
I20260811 06:26:55.453235 24628 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:55.453382 24628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:55.453697 24628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:55.456562 24628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "010474aa2a5045da87d84a24c25251a8"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "514fcef6ae7d1d8b6ebcb0be796a8133"
server_key_iv: "26e155907b8b727ba6a1df7724671e28"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.457413 24628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "010474aa2a5045da87d84a24c25251a8"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "514fcef6ae7d1d8b6ebcb0be796a8133"
server_key_iv: "26e155907b8b727ba6a1df7724671e28"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.462314 24628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:26:55.465554 24644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.466769 24628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:26:55.466940 24628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "010474aa2a5045da87d84a24c25251a8"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "514fcef6ae7d1d8b6ebcb0be796a8133"
server_key_iv: "26e155907b8b727ba6a1df7724671e28"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.467127 24628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:55.480012 24628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:55.483644 24628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:55.483932 24628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:55.484773 24628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:55.486004 24628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:55.486109 24628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.486250 24628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:55.486327 24628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.519835 24628 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34449
I20260811 06:26:55.519884 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34449 every 8 connection(s)
I20260811 06:26:55.521032 24628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:26:55.524147 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24628
I20260811 06:26:55.524384 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:26:55.524838 7186 external_mini_cluster.cc:1428] Setting key 7b65e4dc845737a144969a945340ab19
Aug 11 06:26:55 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:55.542680 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:55.523529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:41427 (local address 127.7.4.190:45487)
0811 06:26:55.523705 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:55.523709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:55.524968 (+ 1259us) server_negotiation.cc:407] Connection header received
0811 06:26:55.527066 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:55.527072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:55.527156 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:55.527273 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:55.528744 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.529913 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.531624 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.532367 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.532510 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:55.536378 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:55.536383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:55.536386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:55.536442 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:55.539104 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:55.540077 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:55.540085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:55.540090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:55.540254 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:55.540721 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:55.540728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:55.540733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:55.541148 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:55.541301 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:55.541954 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 06:26:55.542213 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:26:55.543231 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:55.523943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:41427)
0811 06:26:55.524797 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:55.524849 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:55.526736 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:55.527519 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:55.527527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:55.528025 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:55.528565 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:55.528582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.530068 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.530072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:55.531446 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:55.531458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:55.532565 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:55.532570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:55.532761 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:55.533431 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:55.533453 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:55.536137 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:55.539258 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:55.539269 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:55.539305 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:55.539938 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:55.540366 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:55.540369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:55.540371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:55.540601 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:55.541282 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:55.541288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:55.541588 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 06:26:55.541800 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 06:26:55.542153 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":395,"threads_started":1}
I20260811 06:26:55.544723 24772 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:26:55.544991 24772 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:55.545632 24772 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:26:55.547021 24405 ts_manager.cc:194] Registered new tserver with Master: 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
I20260811 06:26:55.547819 24405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:41427
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:26:55.592517 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:26:55.918254 24777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:26:55.918542 24777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:26:55.918720 24777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:26:55.925738 24777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:26:55.925875 24777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:26:55.926075 24777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:26:55.935566 24777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:26:55.936825 24777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:26:55.938155 24777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:26:55.951403 24782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:55.951349 24783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:26:55.951352 24785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:26:55.952183 24777 server_base.cc:1034] running on GCE node
I20260811 06:26:55.952801 24777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:26:55.953617 24777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:26:55.954864 24777 hybrid_clock.cc:648] HybridClock initialized: now 1786429615954826 us; error 49 us; skew 500 ppm
I20260811 06:26:55.955164 24777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:26:55 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:26:55.958503 24777 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:26:55.960017 24777 webserver.cc:460] Webserver started at http://127.7.4.131:44951/ using document root <none> and password file <none>
I20260811 06:26:55.960666 24777 fs_manager.cc:359] Metadata directory not provided
I20260811 06:26:55.960821 24777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:26:55.961133 24777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:26:55.963939 24777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "0cadf7b4d6245a99f23ee53871ed4156"
server_key_iv: "caccc1b189662d38c01d3f7aeccb7eec"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.964735 24777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "0cadf7b4d6245a99f23ee53871ed4156"
server_key_iv: "caccc1b189662d38c01d3f7aeccb7eec"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.969619 24777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:26:55.972982 24793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.974169 24777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:26:55.974347 24777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "0cadf7b4d6245a99f23ee53871ed4156"
server_key_iv: "caccc1b189662d38c01d3f7aeccb7eec"
server_key_version: "encryptionkey@0"
I20260811 06:26:55.974532 24777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:26:55.991962 24777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:26:55.995539 24777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:26:55.995820 24777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:26:55.996709 24777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:26:55.997926 24777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:26:55.998031 24777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:55.998180 24777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:26:55.998256 24777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:26:56.031549 24777 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:40161
I20260811 06:26:56.031603 24922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:40161 every 8 connection(s)
I20260811 06:26:56.032814 24777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:26:56.034013 24622 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:26:56.038048 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24777
I20260811 06:26:56.038249 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:26:56.038713 7186 external_mini_cluster.cc:1428] Setting key 2687dd9efc0e70b3d814cf125bc76b7c
Aug 11 06:26:56 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:26:56.052896 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.035501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39697 (local address 127.7.4.190:45487)
0811 06:26:56.035710 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.035717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.036868 (+ 1151us) server_negotiation.cc:407] Connection header received
0811 06:26:56.038978 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.038985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.039110 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.039223 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:56.041174 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.042150 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.043656 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.044037 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.044131 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.047230 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:56.047234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:56.047237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:56.047269 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:56.050096 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.050742 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.050745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.050748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.050888 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.051245 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.051248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.051250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.051565 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:56.051679 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.052290 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.052503 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:26:56.054010 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.035976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:39697)
0811 06:26:56.036717 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.036763 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.038642 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.039509 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.039522 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.040282 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:26:56.041005 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.041030 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.042286 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.042290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.043539 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.043550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.044223 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.044228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.044404 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.044983 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:26:56.045008 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:26:56.047049 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:26:56.050224 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:56.050230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:56.050239 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:56.050629 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:56.050978 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:26:56.050981 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:26:56.050984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:26:56.051147 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:26:56.051673 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:26:56.051678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:26:56.051953 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.052157 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.052624 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":283,"threads_started":1}
I20260811 06:26:56.055266 24924 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:26:56.055557 24924 heartbeater.cc:464] Registering TS with master...
I20260811 06:26:56.056181 24924 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:26:56.057475 24405 ts_manager.cc:194] Registered new tserver with Master: 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:26:56.058314 24405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:39697
I20260811 06:26:56.066308 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:26:56.083498 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.071067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.7.4.190:45487)
0811 06:26:56.071264 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.071268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.071778 (+ 510us) server_negotiation.cc:407] Connection header received
0811 06:26:56.071955 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.071959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.072016 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.072095 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:56.073577 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.074264 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.075612 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.076032 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.076137 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.078733 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:56.078738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:56.078741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:56.078777 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:56.081247 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.081877 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.081880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.081882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.082004 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.082339 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.082342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.082344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.082618 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:56.082803 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.083052 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.083200 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:26:56.087152 24405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:26:56.093828 24405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:26:56.135133 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.105223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:43138)
0811 06:26:56.124424 (+ 19201us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.124479 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.124679 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.126781 (+ 2102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.126788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.126831 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:56.127346 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.127362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.129898 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.129903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.132323 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.132343 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.134084 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.134289 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.134342 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.134466 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.134658 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18874,"mutex_wait_us":86,"thread_start_us":21387,"threads_started":1}
I20260811 06:26:56.135232 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.105435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.7.4.129:39133)
0811 06:26:56.126064 (+ 20629us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.126076 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.126109 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:26:56.126220 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.126231 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.126491 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.127576 (+ 1085us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:56.127868 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.129730 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.132489 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.133885 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134005 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.134244 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.134692 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.134877 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20450,"thread_start_us":20292,"threads_started":1}
I20260811 06:26:56.135444 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.107125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56928 (local address 127.7.4.130:34449)
0811 06:26:56.114238 (+ 7113us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.114247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.127781 (+ 13534us) server_negotiation.cc:407] Connection header received
0811 06:26:56.128119 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.128127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.128340 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.128509 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:56.129445 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.131030 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.133239 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.134317 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134464 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.134636 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.134905 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.135075 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6912,"thread_start_us":7149,"threads_started":1}
I20260811 06:26:56.136338 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.106128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39706 (local address 127.7.4.131:40161)
0811 06:26:56.126446 (+ 20318us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.126455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.127042 (+ 587us) server_negotiation.cc:407] Connection header received
0811 06:26:56.127209 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.127219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.127477 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.127666 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:56.128372 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.129902 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.132374 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.133869 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134005 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.134244 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.135811 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.135977 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20143,"thread_start_us":19971,"threads_started":1}
I20260811 06:26:56.135455 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.106430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34449 (local address 127.0.0.1:56928)
0811 06:26:56.127654 (+ 21224us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.127681 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.127893 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.128818 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.128823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.128849 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:56.129268 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.129280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.131171 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.131176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.133089 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.133100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.134511 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.134703 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.134736 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.134841 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.134957 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21076,"mutex_wait_us":32,"thread_start_us":18469,"threads_started":1}
I20260811 06:26:56.137029 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.105305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40161 (local address 127.0.0.1:39706)
0811 06:26:56.126898 (+ 21593us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.126922 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.127058 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.127652 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.127657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.127683 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:56.128134 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.128147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.130092 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.130097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.132208 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.132220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.134438 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.134446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.135522 (+ 1076us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.135559 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.135654 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.136623 (+ 969us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21457,"mutex_wait_us":920,"thread_start_us":22043,"threads_started":1}
I20260811 06:26:56.139948 24701 tablet_service.cc:1467] Processing CreateTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 (DEFAULT_TABLE table=test-table [id=99a0e2691be143d694d2135955102e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:56.139948 24548 tablet_service.cc:1467] Processing CreateTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 (DEFAULT_TABLE table=test-table [id=99a0e2691be143d694d2135955102e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:56.141435 24701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b6e8511bd942aeb7756c231f77f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:56.141431 24548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b6e8511bd942aeb7756c231f77f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:56.142653 24845 tablet_service.cc:1467] Processing CreateTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 (DEFAULT_TABLE table=test-table [id=99a0e2691be143d694d2135955102e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:26:56.143790 24845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b6e8511bd942aeb7756c231f77f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:26:56.155740 24944 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Bootstrap starting.
I20260811 06:26:56.156491 24945 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Bootstrap starting.
I20260811 06:26:56.159876 24945 tablet_bootstrap.cc:654] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:56.159876 24944 tablet_bootstrap.cc:654] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:56.161168 24945 log.cc:824] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:56.161166 24944 log.cc:824] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:56.162510 24946 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Bootstrap starting.
I20260811 06:26:56.164441 24944 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: No bootstrap required, opened a new log
I20260811 06:26:56.164439 24945 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: No bootstrap required, opened a new log
I20260811 06:26:56.164705 24944 ts_tablet_manager.cc:1397] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:26:56.164705 24945 ts_tablet_manager.cc:1397] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:26:56.165163 24946 tablet_bootstrap.cc:654] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Neither blocks nor log segments found. Creating new log.
I20260811 06:26:56.166074 24946 log.cc:824] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:26:56.169770 24946 tablet_bootstrap.cc:492] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: No bootstrap required, opened a new log
I20260811 06:26:56.170010 24946 ts_tablet_manager.cc:1397] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 06:26:56.175207 24944 raft_consensus.cc:348] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.175681 24944 raft_consensus.cc:374] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:56.175800 24944 raft_consensus.cc:729] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f4d2c7d0c90481f9c5d88dc199f6c16, State: Initialized, Role: FOLLOWER
I20260811 06:26:56.176054 24945 raft_consensus.cc:348] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.176440 24945 raft_consensus.cc:374] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:56.176535 24945 raft_consensus.cc:729] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb5fe222a954fb99a0d5c81a27ec24b, State: Initialized, Role: FOLLOWER
I20260811 06:26:56.176434 24944 consensus_queue.cc:260] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.177155 24945 consensus_queue.cc:260] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.177662 24946 raft_consensus.cc:348] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.178093 24946 raft_consensus.cc:374] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:26:56.178205 24946 raft_consensus.cc:729] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010474aa2a5045da87d84a24c25251a8, State: Initialized, Role: FOLLOWER
I20260811 06:26:56.178743 24946 consensus_queue.cc:260] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.179546 24945 ts_tablet_manager.cc:1428] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 06:26:56.180704 24772 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:26:56.181089 24946 ts_tablet_manager.cc:1428] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 06:26:56.183519 24924 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:26:56.184428 24944 ts_tablet_manager.cc:1428] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Time spent starting tablet: real 0.020s user 0.020s sys 0.000s
I20260811 06:26:56.209401 24952 raft_consensus.cc:482] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:26:56.209661 24952 raft_consensus.cc:504] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.211324 24952 leader_election.cc:288] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133), 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:26:56.217722 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.211770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:48551 (local address 127.7.4.131:40161)
0811 06:26:56.211965 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.211974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.212442 (+ 468us) server_negotiation.cc:407] Connection header received
0811 06:26:56.212683 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.212687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.212743 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.212838 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:56.213390 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.214134 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.216042 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.216791 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.216948 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.217054 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.217345 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.217469 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 06:26:56.217882 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.211554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40161 (local address 127.7.4.130:48551)
0811 06:26:56.212328 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.212344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.212532 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.212869 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.212874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.212902 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:56.213239 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.213251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.214268 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.214277 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.215883 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.215895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.217013 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.217019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.217175 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.217199 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.217314 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.217452 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":3,"thread_start_us":440,"threads_started":1}
I20260811 06:26:56.217954 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.211853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:38227 (local address 127.7.4.129:39133)
0811 06:26:56.212105 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.212113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.212762 (+ 649us) server_negotiation.cc:407] Connection header received
0811 06:26:56.212957 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.212962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.213026 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.213120 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:26:56.213719 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.214413 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.216213 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.216999 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.217155 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.217260 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.217560 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.217694 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:26:56.218353 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.211629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.7.4.130:38227)
0811 06:26:56.212646 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0811 06:26:56.212660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:26:56.212811 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:26:56.213148 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:26:56.213152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:26:56.213189 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:26:56.213552 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.213561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.214568 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.214575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.216050 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:26:56.216061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.217234 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.217242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:26:56.217402 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.217425 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:26:56.217530 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:26:56.217654 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":21,"thread_start_us":677,"threads_started":1}
I20260811 06:26:56.218997 24868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b6e8511bd942aeb7756c231f77f3b4" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" is_pre_election: true
I20260811 06:26:56.219350 24572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b6e8511bd942aeb7756c231f77f3b4" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" is_pre_election: true
I20260811 06:26:56.219520 24868 raft_consensus.cc:2454] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 0.
I20260811 06:26:56.219822 24572 raft_consensus.cc:2454] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 0.
I20260811 06:26:56.220211 24646 leader_election.cc:302] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16; no voters:
I20260811 06:26:56.220647 24952 raft_consensus.cc:2781] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:26:56.220803 24952 raft_consensus.cc:482] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:26:56.220922 24952 raft_consensus.cc:3037] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:56.223130 24952 raft_consensus.cc:504] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.223755 24952 leader_election.cc:288] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133), 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:26:56.224198 24572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b6e8511bd942aeb7756c231f77f3b4" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
I20260811 06:26:56.224239 24868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b6e8511bd942aeb7756c231f77f3b4" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
I20260811 06:26:56.224400 24572 raft_consensus.cc:3037] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:56.224443 24868 raft_consensus.cc:3037] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:26:56.226564 24572 raft_consensus.cc:2454] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 1.
I20260811 06:26:56.226655 24868 raft_consensus.cc:2454] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 1.
I20260811 06:26:56.227015 24647 leader_election.cc:302] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010474aa2a5045da87d84a24c25251a8, 4cb5fe222a954fb99a0d5c81a27ec24b; no voters:
I20260811 06:26:56.227393 24952 raft_consensus.cc:2781] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:26:56.227890 24952 raft_consensus.cc:686] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 1 LEADER]: Becoming Leader. State: Replica: 010474aa2a5045da87d84a24c25251a8, State: Running, Role: LEADER
I20260811 06:26:56.228459 24952 consensus_queue.cc:237] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:26:56.233294 24404 catalog_manager.cc:5495] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 010474aa2a5045da87d84a24c25251a8 (127.7.4.130). New cstate: current_term: 1 leader_uuid: "010474aa2a5045da87d84a24c25251a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } health_report { overall_health: HEALTHY } } }
W20260811 06:26:56.256507 24623 tablet.cc:2369] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:56.271165 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.265468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.7.4.130:34449)
0811 06:26:56.265690 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.265696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.265716 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:26:56.265976 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.265980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.266060 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.266220 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:26:56.266605 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.267409 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.268926 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.269347 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.269487 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.269988 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:26:56.270527 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:26:56.270680 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.270797 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.270879 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 06:26:56.272998 24773 tablet.cc:2369] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:56.279115 24572 raft_consensus.cc:1261] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Refusing update from remote peer 010474aa2a5045da87d84a24c25251a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:56.279117 24868 raft_consensus.cc:1261] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Refusing update from remote peer 010474aa2a5045da87d84a24c25251a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:26:56.280146 24956 consensus_queue.cc:1035] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:26:56.280490 24952 consensus_queue.cc:1035] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:26:56.284732 24925 tablet.cc:2369] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:26:56.294991 24963 mvcc.cc:205] Tried to move back new op lower bound from 7317215708269072384 to 7317215708076089344. Current Snapshot: MvccSnapshot[applied={T|T < 7317215708269072384}]
I20260811 06:26:56.307585 24404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:56.307865 24404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:26:56.311550 24404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Sending DeleteTablet for 3 replicas of tablet e0b6e8511bd942aeb7756c231f77f3b4
I20260811 06:26:56.312624 24548 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:56 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43138
I20260811 06:26:56.312711 24845 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:56 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:39706
I20260811 06:26:56.312769 24701 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b6e8511bd942aeb7756c231f77f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:26:56 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:56928
I20260811 06:26:56.313901 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24475
I20260811 06:26:56.327926 24970 tablet_replica.cc:330] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: stopping tablet replica
I20260811 06:26:56.328457 24970 raft_consensus.cc:2229] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:56.329073 24970 raft_consensus.cc:2258] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:56.331866 24970 ts_tablet_manager.cc:1905] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:56.340538 24970 ts_tablet_manager.cc:1918] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:56.340718 24970 log.cc:1196] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/e0b6e8511bd942aeb7756c231f77f3b4
I20260811 06:26:56.341284 24970 ts_tablet_manager.cc:1939] T e0b6e8511bd942aeb7756c231f77f3b4 P 4cb5fe222a954fb99a0d5c81a27ec24b: Deleting consensus metadata
I20260811 06:26:56.343327 24385 catalog_manager.cc:4841] TS 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133): tablet e0b6e8511bd942aeb7756c231f77f3b4 (table test-table [id=99a0e2691be143d694d2135955102e16]) successfully deleted
I20260811 06:26:56.346520 24971 tablet_replica.cc:330] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: stopping tablet replica
I20260811 06:26:56.346779 24972 tablet_replica.cc:330] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: stopping tablet replica
I20260811 06:26:56.346978 24971 raft_consensus.cc:2229] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:26:56.347221 24972 raft_consensus.cc:2229] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:26:56.347546 24971 raft_consensus.cc:2258] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:56.348042 24972 raft_consensus.cc:2258] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:26:56.350102 24971 ts_tablet_manager.cc:1905] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:26:56.350189 24972 ts_tablet_manager.cc:1905] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:26:56 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:26:56.357329 24971 ts_tablet_manager.cc:1918] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:56.357487 24972 ts_tablet_manager.cc:1918] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:26:56.357515 24971 log.cc:1196] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/e0b6e8511bd942aeb7756c231f77f3b4
I20260811 06:26:56.357712 24972 log.cc:1196] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/e0b6e8511bd942aeb7756c231f77f3b4
I20260811 06:26:56.358192 24971 ts_tablet_manager.cc:1939] T e0b6e8511bd942aeb7756c231f77f3b4 P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting consensus metadata
I20260811 06:26:56.358304 24972 ts_tablet_manager.cc:1939] T e0b6e8511bd942aeb7756c231f77f3b4 P 010474aa2a5045da87d84a24c25251a8: Deleting consensus metadata
I20260811 06:26:56.360162 24386 catalog_manager.cc:4841] TS 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): tablet e0b6e8511bd942aeb7756c231f77f3b4 (table test-table [id=99a0e2691be143d694d2135955102e16]) successfully deleted
I20260811 06:26:56.360492 24384 catalog_manager.cc:4841] TS 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): tablet e0b6e8511bd942aeb7756c231f77f3b4 (table test-table [id=99a0e2691be143d694d2135955102e16]) successfully deleted
I20260811 06:26:56.361560 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.315992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.7.4.129:39133)
0811 06:26:56.316211 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.316220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.340571 (+ 24351us) server_negotiation.cc:407] Connection header received
0811 06:26:56.341159 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.341166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.341253 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.341347 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:56.343620 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.344408 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.346603 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.347202 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.347359 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.353553 (+ 6194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:56.353568 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:56.353585 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:56.353635 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:56.357515 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.358675 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.358698 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.358701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.358924 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.359463 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.359468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.359474 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.360013 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:56.360316 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.360960 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.361183 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:26:56.362323 24617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:56.467159 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24475
I20260811 06:26:56.490272 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24628
Aug 11 06:26:56 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:26:56.513000 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.491065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.7.4.130:34449)
0811 06:26:56.491266 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.491271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.491332 (+ 61us) server_negotiation.cc:407] Connection header received
0811 06:26:56.491527 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.491531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.491602 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.491689 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:56.493436 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.494299 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.496386 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.496873 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.496997 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.506761 (+ 9764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:56.506775 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:56.506789 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:56.506827 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:56.509943 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.510659 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.510666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.510669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.510807 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.511241 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.511245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.511247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.511712 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:56.512086 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.512404 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.512627 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:26:56.513672 24766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:56.629791 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24628
I20260811 06:26:56.653697 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24777
Aug 11 06:26:56 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:26:56.669044 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:26:56.654464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.7.4.131:40161)
0811 06:26:56.654634 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:26:56.654638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:26:56.654680 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:26:56.654857 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:26:56.654861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:26:56.654927 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:26:56.655011 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:26:56.656405 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.657108 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.658612 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:26:56.658999 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:26:56.659092 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:26:56.663412 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:26:56.663426 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:26:56.663441 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:26:56.663478 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:26:56.666295 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.666922 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.666929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.666931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.667071 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:26:56.667656 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:26:56.667659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:26:56.667661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:26:56.667994 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:26:56.668284 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 06:26:56.668629 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 06:26:56.668799 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:26:56.669739 24918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:56.758965 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24777
I20260811 06:26:56.779217 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24363
I20260811 06:26:56.780366 24443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:26:56.891335 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24363
I20260811 06:27:16.917937 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--webserver_interface=127.7.4.190
--webserver_port=40765
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:17.271427 24978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:17.271739 24978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:17.271907 24978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:17.279071 24978 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:27:17.279173 24978 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:27:17.279253 24978 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:27:17.279373 24978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:27:17.279469 24978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:17.279605 24978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:17.279685 24978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:27:17.279776 24978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:27:17.289595 24978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=40765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:17.290917 24978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:17.292263 24978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:17.317999 24985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:17.318013 24986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:17.318634 24989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:17.319507 24978 server_base.cc:1034] running on GCE node
I20260811 06:27:17.320431 24978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:17.321992 24978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:17.323340 24978 hybrid_clock.cc:648] HybridClock initialized: now 1786429637323207 us; error 38 us; skew 500 ppm
I20260811 06:27:17.323781 24978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:17 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:17.331931 24978 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:27:17.333694 24978 webserver.cc:460] Webserver started at http://127.7.4.190:40765/ using document root <none> and password file <none>
I20260811 06:27:17.334494 24978 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:17.334648 24978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:17.340543 24978 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:27:17.343429 24997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:17.344769 24978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:27:17.344947 24978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "5478333bcb9c48ec8bee9d121d5ae088"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "492a719dcf839c4507385287bb47dc9e"
server_key_iv: "4289f58ebeabf0007e5a114d049436b0"
server_key_version: "encryptionkey@0"
I20260811 06:27:17.346231 24978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:17.360945 24978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:17.364722 24978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:17.364960 24978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:17.384227 24978 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45487
I20260811 06:27:17.384251 25059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45487 every 8 connection(s)
I20260811 06:27:17.385759 24978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:27:17.390777 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 24978
I20260811 06:27:17.391848 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:39133
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=34863
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:27:17.393635 25060 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:27:17.398847 25060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap starting.
I20260811 06:27:17.420554 25060 log.cc:824] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:17.432731 25060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:27:17.433287 25060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap complete.
I20260811 06:27:17.442420 25060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:17.443236 25060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Initialized, Role: FOLLOWER
I20260811 06:27:17.443914 25060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:17.444161 25060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:27:17.444314 25060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:27:17.444497 25060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:27:17.448387 25060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:17.448912 25060 leader_election.cc:302] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5478333bcb9c48ec8bee9d121d5ae088; no voters:
I20260811 06:27:17.449661 25060 leader_election.cc:288] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:27:17.449810 25064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:27:17.451560 25064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 LEADER]: Becoming Leader. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Running, Role: LEADER
I20260811 06:27:17.452121 25064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:17.452312 25060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:27:17.454794 25065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:27:17.455214 25065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:17.456966 25069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:27:17.456858 25066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5478333bcb9c48ec8bee9d121d5ae088. Latest consensus state: current_term: 2 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:27:17.457248 25066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:17.465169 25069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:27:17.466527 25069 catalog_manager.cc:1259] Loaded cluster ID: b4999faa55924498a86ac2789ba0b8b1
I20260811 06:27:17.466667 25069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:27:17.470854 25069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:27:17.472566 25069 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Loaded TSK: 0
I20260811 06:27:17.515870 25069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Generated new TSK 1
I20260811 06:27:17.516263 25069 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:27:17.756047 25062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:17.756356 25062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:17.756525 25062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:17.764380 25062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:17.764523 25062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:17.764711 25062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:27:17.774708 25062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:39133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=34863
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:17.776141 25062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:17.777513 25062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:17.792500 25089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:17.792798 25092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:17.792515 25090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:17.793853 25062 server_base.cc:1034] running on GCE node
I20260811 06:27:17.794564 25062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:17.795666 25062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:17.796888 25062 hybrid_clock.cc:648] HybridClock initialized: now 1786429637796821 us; error 62 us; skew 500 ppm
I20260811 06:27:17.797199 25062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:17 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:17.800516 25062 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:27:17.802104 25062 webserver.cc:460] Webserver started at http://127.7.4.129:34863/ using document root <none> and password file <none>
I20260811 06:27:17.802773 25062 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:17.802932 25062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:17.808235 25062 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:27:17.810972 25100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:17.812325 25062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:27:17.812485 25062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "8931e35c759180f6b50feb464e93c057"
server_key_iv: "a10f065c9255274d828f05e9c8127fcf"
server_key_version: "encryptionkey@0"
I20260811 06:27:17.813498 25062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:17.839183 25062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:17.843232 25062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:17.843545 25062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:17.844638 25062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:17.845919 25062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:17.846055 25062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:17.846169 25062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:17.846263 25062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:17.880822 25062 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39133
I20260811 06:27:17.880934 25228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39133 every 8 connection(s)
I20260811 06:27:17.882272 25062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:27:17.884982 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25062
I20260811 06:27:17.885766 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:34449
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=43183
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:27:17 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:17.907153 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:17.885358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:44253)
0811 06:27:17.886331 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:17.886375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:17.888185 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:17.889697 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:17.889704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:17.890196 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:17.890692 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:17.890709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:17.893129 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:17.893133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:17.894309 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:17.894321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:17.895801 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:17.895805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:17.895982 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:17.896658 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:17.896679 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:17.898767 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:17.903343 (+ 4576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:17.903350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:17.903359 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:17.903747 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:17.904230 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:17.904233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:17.904242 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:17.904418 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:17.905348 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:17.905353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:17.905629 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 06:27:17.905836 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 06:27:17.906178 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":480,"threads_started":1}
I20260811 06:27:17.907863 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:17.885265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:44253 (local address 127.7.4.190:45487)
0811 06:27:17.886158 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:17.886175 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:17.886516 (+ 341us) server_negotiation.cc:407] Connection header received
0811 06:27:17.888765 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:17.888788 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:17.889197 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:17.889593 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:17.890906 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:17.892951 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:17.894457 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:17.895599 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:17.895805 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:17.899052 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:17.899064 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:17.899085 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:17.899141 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:17.903140 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:17.903883 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:17.903893 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:17.903903 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:17.904112 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:17.904534 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:17.904542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:17.904548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:17.905193 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:17.905393 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:17.906103 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 06:27:17.906609 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":258,"threads_started":1}
I20260811 06:27:17.910589 25229 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:27:17.910936 25229 heartbeater.cc:464] Registering TS with master...
I20260811 06:27:17.911854 25229 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:27:17.914021 25017 ts_manager.cc:194] Registered new tserver with Master: 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133)
I20260811 06:27:17.915499 25017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:44253
W20260811 06:27:18.232051 25233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:18.232312 25233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:18.232489 25233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:18.239332 25233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:18.239450 25233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:18.239629 25233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:27:18.248953 25233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:34449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=43183
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:18.250197 25233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:18.251556 25233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:18.264972 25238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:18.265089 25239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:18.265179 25242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:18.266161 25233 server_base.cc:1034] running on GCE node
I20260811 06:27:18.266707 25233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:18.267683 25233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:18.268930 25233 hybrid_clock.cc:648] HybridClock initialized: now 1786429638268895 us; error 48 us; skew 500 ppm
I20260811 06:27:18.269234 25233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:18 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:18.272403 25233 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:27:18.273885 25233 webserver.cc:460] Webserver started at http://127.7.4.130:43183/ using document root <none> and password file <none>
I20260811 06:27:18.274542 25233 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:18.274698 25233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:18.279717 25233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:27:18.282199 25250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.283490 25233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:27:18.283663 25233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "010474aa2a5045da87d84a24c25251a8"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "514fcef6ae7d1d8b6ebcb0be796a8133"
server_key_iv: "26e155907b8b727ba6a1df7724671e28"
server_key_version: "encryptionkey@0"
I20260811 06:27:18.284624 25233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:18.336063 25233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:18.339991 25233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:18.340270 25233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:18.341245 25233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:18.342552 25233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:18.342685 25233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.342836 25233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:18.342929 25233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.377497 25233 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34449
I20260811 06:27:18.377579 25380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34449 every 8 connection(s)
I20260811 06:27:18.378947 25233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:27:18.382211 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25233
I20260811 06:27:18.383025 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:40161
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=44951
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:27:18 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:18.405442 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.381567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:41107 (local address 127.7.4.190:45487)
0811 06:27:18.381738 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.381743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.383148 (+ 1405us) server_negotiation.cc:407] Connection header received
0811 06:27:18.385812 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.385818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.385923 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.386028 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:18.388224 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.389429 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.391752 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.392522 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.392661 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.397959 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:18.397964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:18.397967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:18.397999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:18.401757 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.402661 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.402687 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.402693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.402850 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.403400 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.403408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.403414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.403805 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:18.403966 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.404726 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.404983 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:27:18.406001 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.382204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:41107)
0811 06:27:18.383001 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:18.383039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:18.385466 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:18.386519 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:18.386529 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:18.387210 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:18.388007 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.388031 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.389603 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.389609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.391594 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.391609 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.392749 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.392755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.393037 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.393817 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:18.393849 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:18.396746 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:18.401911 (+ 5165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:18.401923 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:18.401940 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:18.402502 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:18.402973 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:18.402978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:18.402982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:18.403242 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:18.403948 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:18.403956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:18.404345 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 06:27:18.404578 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 06:27:18.404940 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":321,"threads_started":1}
I20260811 06:27:18.407477 25382 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:27:18.407816 25382 heartbeater.cc:464] Registering TS with master...
I20260811 06:27:18.408697 25382 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:27:18.410267 25017 ts_manager.cc:194] Registered new tserver with Master: 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
I20260811 06:27:18.411175 25017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:41107
W20260811 06:27:18.719949 25385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:18.720220 25385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:18.720382 25385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:18.727275 25385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:18.727464 25385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:18.727646 25385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:27:18.736785 25385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:40161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=44951
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:18.737902 25385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:18.739096 25385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:18.751747 25390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:18.751722 25391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:18.752134 25393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:18.752844 25385 server_base.cc:1034] running on GCE node
I20260811 06:27:18.753417 25385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:18.754330 25385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:18.755544 25385 hybrid_clock.cc:648] HybridClock initialized: now 1786429638755513 us; error 45 us; skew 500 ppm
I20260811 06:27:18.755827 25385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:18 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:18.758729 25385 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:27:18.760118 25385 webserver.cc:460] Webserver started at http://127.7.4.131:44951/ using document root <none> and password file <none>
I20260811 06:27:18.760680 25385 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:18.760804 25385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:18.765647 25385 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:27:18.768092 25401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.769348 25385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:27:18.769474 25385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "0cadf7b4d6245a99f23ee53871ed4156"
server_key_iv: "caccc1b189662d38c01d3f7aeccb7eec"
server_key_version: "encryptionkey@0"
I20260811 06:27:18.770339 25385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:18.790860 25385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:18.794348 25385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:18.794565 25385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:18.795437 25385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:18.796586 25385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:18.796674 25385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.796749 25385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:18.796802 25385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:18.828827 25385 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:40161
I20260811 06:27:18.828907 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:40161 every 8 connection(s)
I20260811 06:27:18.830171 25385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:27:18.838948 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25385
Aug 11 06:27:18 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:18.851105 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.832731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:59873 (local address 127.7.4.190:45487)
0811 06:27:18.832920 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.832925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.834017 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 06:27:18.835883 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.835889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.835971 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.836088 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:18.837561 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.838277 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.840123 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.840831 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.840961 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.844794 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:18.844798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:18.844800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:18.844829 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:18.847604 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.848460 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.848463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.848465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.848620 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.849095 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.849098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.849100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.849379 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:18.849562 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.850405 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.850651 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:27:18.852131 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.833167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:59873)
0811 06:27:18.833856 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:18.833899 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:18.835611 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:18.836337 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:18.836345 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:18.836808 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:18.837373 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.837390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.838432 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.838442 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.839941 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.839955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.841073 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.841080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.841404 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.842059 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:18.842087 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:18.844623 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:18.847731 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:18.847739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:18.847754 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:18.848329 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:18.848756 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:18.848761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:18.848765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:18.848956 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:18.849532 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:18.849557 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:18.850014 (+ 457us) client_negotiation.cc:769] Sending connection context
0811 06:27:18.850297 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 06:27:18.850813 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":265,"threads_started":1}
I20260811 06:27:18.853600 25529 heartbeater.cc:347] Connected to a master server at 127.7.4.190:45487
I20260811 06:27:18.853907 25529 heartbeater.cc:464] Registering TS with master...
I20260811 06:27:18.854588 25529 heartbeater.cc:511] Master 127.7.4.190:45487 requested a full tablet report, sending...
I20260811 06:27:18.855021 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.839915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.7.4.190:45487)
0811 06:27:18.840460 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.840466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.840805 (+ 339us) server_negotiation.cc:407] Connection header received
0811 06:27:18.840977 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.840982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.841065 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.841185 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:18.842759 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.843800 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.845475 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.845950 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.846104 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.848878 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:18.848884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:18.848890 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:18.848921 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:18.852229 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.852948 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.852954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.852957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.853075 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.853477 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.853483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.853486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.853791 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:18.854051 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.854380 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.854592 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":283,"threads_started":1}
I20260811 06:27:18.856110 25016 ts_manager.cc:194] Registered new tserver with Master: 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:27:18.856957 25016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:59873
I20260811 06:27:18.859323 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:27:18.865306 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.860308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.7.4.190:45487)
0811 06:27:18.860515 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.860523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.860897 (+ 374us) server_negotiation.cc:407] Connection header received
0811 06:27:18.861068 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.861071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.861173 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.861278 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:27:18.861666 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.862309 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.863801 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.864179 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.864268 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.864671 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:27:18.864892 (+ 221us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 06:27:18.865134 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36704: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 06:27:18.865419 25532 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36704: token expired
W20260811 06:27:18.865984 24928 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.4.190:45487)
I20260811 06:27:18.877974 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.866398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.7.4.190:45487)
0811 06:27:18.866535 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.866539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.866553 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:27:18.866671 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.866674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.866719 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.866794 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:18.868271 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.869048 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.870345 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.870705 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.870800 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.873391 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:18.873395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:18.873398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:18.873424 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:18.875877 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.876465 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.876468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.876470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.876578 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:18.876878 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:18.876881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:18.876883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:18.877137 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:18.877299 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.877543 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.877679 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:27:18.880931 25016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:27:18.887804 25016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:27:18.919239 25229 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:27:18.919685 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.898807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40161 (local address 127.0.0.1:51288)
0811 06:27:18.910343 (+ 11536us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:18.910367 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:18.910527 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:18.911273 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:18.911280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:18.911331 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:27:18.911839 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.911855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.914280 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.914288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.917242 (+ 2954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.917252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.918677 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.918683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.918881 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.918936 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 06:27:18.919060 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 06:27:18.919240 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11376,"mutex_wait_us":166,"thread_start_us":11231,"threads_started":1}
I20260811 06:27:18.920006 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.899430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34449 (local address 127.0.0.1:38090)
0811 06:27:18.910680 (+ 11250us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:18.910697 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:18.910814 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:18.911938 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:18.911944 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:18.911976 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:27:18.912548 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.912565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.914165 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.914169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.916985 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.916998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.918995 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.919002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.919255 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.919390 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 06:27:18.919476 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 06:27:18.919606 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11105,"mutex_wait_us":922,"thread_start_us":10322,"threads_started":1}
I20260811 06:27:18.920049 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.898625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.7.4.129:39133)
0811 06:27:18.911144 (+ 12519us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.911153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.911450 (+ 297us) server_negotiation.cc:407] Connection header received
0811 06:27:18.911557 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.911565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.911789 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.912200 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:27:18.912880 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.914370 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.916999 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.918489 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.918611 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.918833 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.919573 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.919749 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12343,"thread_start_us":12208,"threads_started":1}
I20260811 06:27:18.920571 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.898448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:43978)
0811 06:27:18.899680 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:18.899723 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:18.899889 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:18.911992 (+ 12103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:18.911998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:18.912022 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:27:18.912692 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.912706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.914513 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.914521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.916816 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:18.916851 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.919012 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.919018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:18.919242 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.919388 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 06:27:18.919476 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:27:18.920269 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":930,"spinlock_wait_cycles":1920,"thread_start_us":992,"threads_started":1}
I20260811 06:27:18.921137 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.899830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.7.4.130:34449)
0811 06:27:18.909060 (+ 9230us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.909072 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.910922 (+ 1850us) server_negotiation.cc:407] Connection header received
0811 06:27:18.911029 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.911038 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.911455 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.911652 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:27:18.912743 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.913759 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.917136 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.918500 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.918611 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.918791 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.920767 (+ 1976us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.920925 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9084,"thread_start_us":8986,"threads_started":1}
I20260811 06:27:18.922444 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:18.899495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.7.4.131:40161)
0811 06:27:18.910005 (+ 10510us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:18.910015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:18.910664 (+ 649us) server_negotiation.cc:407] Connection header received
0811 06:27:18.910779 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:18.910786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:18.911018 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:18.912088 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:27:18.912457 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.914125 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.917383 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:18.918461 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:18.918607 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:18.918844 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:18.921980 (+ 3136us) server_negotiation.cc:299] Negotiation successful
0811 06:27:18.922148 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10342,"thread_start_us":10203,"threads_started":1}
I20260811 06:27:18.924552 25153 tablet_service.cc:1467] Processing CreateTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff (DEFAULT_TABLE table=test-table [id=d1ba28239dce4ddfb95c58f3a968d9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:27:18.924700 25305 tablet_service.cc:1467] Processing CreateTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff (DEFAULT_TABLE table=test-table [id=d1ba28239dce4ddfb95c58f3a968d9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:27:18.925808 25153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b8c5a812d245f4ac06d3fe10b50bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:27:18.926045 25305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b8c5a812d245f4ac06d3fe10b50bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:27:18.926029 25454 tablet_service.cc:1467] Processing CreateTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff (DEFAULT_TABLE table=test-table [id=d1ba28239dce4ddfb95c58f3a968d9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:27:18.927305 25454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b8c5a812d245f4ac06d3fe10b50bff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:27:18.937028 25549 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Bootstrap starting.
I20260811 06:27:18.940687 25549 tablet_bootstrap.cc:654] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Neither blocks nor log segments found. Creating new log.
I20260811 06:27:18.941445 25551 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Bootstrap starting.
I20260811 06:27:18.941989 25549 log.cc:824] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:18.945165 25551 tablet_bootstrap.cc:654] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Neither blocks nor log segments found. Creating new log.
I20260811 06:27:18.945359 25549 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: No bootstrap required, opened a new log
I20260811 06:27:18.945622 25549 ts_tablet_manager.cc:1397] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 06:27:18.946334 25551 log.cc:824] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:18.947420 25553 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Bootstrap starting.
I20260811 06:27:18.949488 25551 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: No bootstrap required, opened a new log
I20260811 06:27:18.949798 25551 ts_tablet_manager.cc:1397] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:27:18.950966 25553 tablet_bootstrap.cc:654] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Neither blocks nor log segments found. Creating new log.
I20260811 06:27:18.952138 25553 log.cc:824] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:18.954342 25549 raft_consensus.cc:348] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.954680 25549 raft_consensus.cc:374] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:27:18.954820 25549 raft_consensus.cc:729] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb5fe222a954fb99a0d5c81a27ec24b, State: Initialized, Role: FOLLOWER
I20260811 06:27:18.955056 25553 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: No bootstrap required, opened a new log
I20260811 06:27:18.955331 25553 ts_tablet_manager.cc:1397] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260811 06:27:18.955452 25549 consensus_queue.cc:260] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.957047 25549 ts_tablet_manager.cc:1428] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260811 06:27:18.959980 25551 raft_consensus.cc:348] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.960426 25551 raft_consensus.cc:374] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:27:18.960529 25551 raft_consensus.cc:729] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010474aa2a5045da87d84a24c25251a8, State: Initialized, Role: FOLLOWER
I20260811 06:27:18.961094 25551 consensus_queue.cc:260] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.962178 25382 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:27:18.962743 25551 ts_tablet_manager.cc:1428] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260811 06:27:18.965337 25553 raft_consensus.cc:348] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.965749 25553 raft_consensus.cc:374] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:27:18.965873 25553 raft_consensus.cc:729] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f4d2c7d0c90481f9c5d88dc199f6c16, State: Initialized, Role: FOLLOWER
I20260811 06:27:18.966473 25553 consensus_queue.cc:260] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:18.967484 25529 heartbeater.cc:503] Master 127.7.4.190:45487 was elected leader, sending a full tablet report...
I20260811 06:27:18.967962 25553 ts_tablet_manager.cc:1428] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260811 06:27:19.000656 25558 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:19.000979 25558 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:19.002549 25558 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133), 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:27:19.010380 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.003087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:43893 (local address 127.7.4.129:39133)
0811 06:27:19.003326 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.003331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.004199 (+ 868us) server_negotiation.cc:407] Connection header received
0811 06:27:19.004342 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.004347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.004420 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.004531 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:27:19.005436 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.006532 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.008607 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.009413 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.009569 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.009683 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.009995 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.010137 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:27:19.010562 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.002908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.7.4.130:43893)
0811 06:27:19.004095 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.004111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:19.004208 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:19.004572 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:19.004577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:19.004634 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:27:19.005267 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:19.005290 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.006728 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.006737 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:19.008438 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:19.008450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.009673 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.009680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:19.009838 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.009864 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:27:19.009967 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:27:19.010102 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":80,"spinlock_wait_cycles":2048,"thread_start_us":621,"threads_started":1}
I20260811 06:27:19.010627 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.002868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40161 (local address 127.7.4.130:60429)
0811 06:27:19.003984 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.004004 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:19.004174 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:19.004572 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:19.004577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:19.004640 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:27:19.005220 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:19.005236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.006673 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.006679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:19.008445 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:19.008458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.009730 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.009738 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:19.009913 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.009936 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:27:19.010039 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:27:19.010171 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":190,"spinlock_wait_cycles":14848,"thread_start_us":710,"threads_started":1}
I20260811 06:27:19.010995 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.003100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:60429 (local address 127.7.4.131:40161)
0811 06:27:19.003320 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.003326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.004107 (+ 781us) server_negotiation.cc:407] Connection header received
0811 06:27:19.004339 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.004344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.004419 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.004531 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:27:19.005399 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.006521 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.008631 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.009489 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.009644 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.009762 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.010079 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.010200 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:27:19.011631 25177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b8c5a812d245f4ac06d3fe10b50bff" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" is_pre_election: true
I20260811 06:27:19.011967 25476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b8c5a812d245f4ac06d3fe10b50bff" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" is_pre_election: true
I20260811 06:27:19.012101 25177 raft_consensus.cc:2454] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 0.
I20260811 06:27:19.012431 25476 raft_consensus.cc:2454] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 0.
I20260811 06:27:19.012872 25253 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010474aa2a5045da87d84a24c25251a8, 4cb5fe222a954fb99a0d5c81a27ec24b; no voters:
I20260811 06:27:19.013310 25558 raft_consensus.cc:2781] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:27:19.013428 25558 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:27:19.013499 25558 raft_consensus.cc:3037] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:27:19.015728 25558 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:19.016302 25558 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133), 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161)
I20260811 06:27:19.016691 25476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b8c5a812d245f4ac06d3fe10b50bff" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
I20260811 06:27:19.016741 25177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b8c5a812d245f4ac06d3fe10b50bff" candidate_uuid: "010474aa2a5045da87d84a24c25251a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
I20260811 06:27:19.016866 25476 raft_consensus.cc:3037] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:27:19.016961 25177 raft_consensus.cc:3037] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:27:19.019271 25476 raft_consensus.cc:2454] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 1.
I20260811 06:27:19.019492 25177 raft_consensus.cc:2454] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010474aa2a5045da87d84a24c25251a8 in term 1.
I20260811 06:27:19.019748 25252 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16; no voters:
I20260811 06:27:19.020028 25558 raft_consensus.cc:2781] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:27:19.020522 25558 raft_consensus.cc:686] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 1 LEADER]: Becoming Leader. State: Replica: 010474aa2a5045da87d84a24c25251a8, State: Running, Role: LEADER
I20260811 06:27:19.020962 25558 consensus_queue.cc:237] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:19.025458 25017 catalog_manager.cc:5495] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 010474aa2a5045da87d84a24c25251a8 (127.7.4.130). New cstate: current_term: 1 leader_uuid: "010474aa2a5045da87d84a24c25251a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } health_report { overall_health: HEALTHY } } }
I20260811 06:27:19.041400 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.036309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.7.4.130:34449)
0811 06:27:19.036509 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.036515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.036533 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:27:19.036743 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.036748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.036835 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.036972 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:27:19.037318 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.037994 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.039319 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.039713 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.039844 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.040282 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:27:19.040779 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:27:19.040942 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.041050 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.041123 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:27:19.049340 25476 raft_consensus.cc:1261] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Refusing update from remote peer 010474aa2a5045da87d84a24c25251a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:27:19.049456 25177 raft_consensus.cc:1261] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Refusing update from remote peer 010474aa2a5045da87d84a24c25251a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:27:19.050339 25565 consensus_queue.cc:1035] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:27:19.050707 25558 consensus_queue.cc:1035] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:27:19.062433 25567 mvcc.cc:205] Tried to move back new op lower bound from 7317215801536335872 to 7317215801433321472. Current Snapshot: MvccSnapshot[applied={T|T < 7317215801536335872}]
I20260811 06:27:19.076162 25017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:27:19.076427 25017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:27:19.079702 25017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Sending DeleteTablet for 3 replicas of tablet 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:19.080740 25454 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:27:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:51288
I20260811 06:27:19.080819 25305 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:27:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:38090
I20260811 06:27:19.080850 25153 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b8c5a812d245f4ac06d3fe10b50bff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:27:19 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:43978
I20260811 06:27:19.081919 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 25062
I20260811 06:27:19.094357 25579 tablet_replica.cc:330] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: stopping tablet replica
I20260811 06:27:19.094782 25579 raft_consensus.cc:2229] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:27:19.095415 25579 raft_consensus.cc:2258] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:27:19.097448 25579 ts_tablet_manager.cc:1905] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:27:19.103754 25578 tablet_replica.cc:330] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: stopping tablet replica
I20260811 06:27:19.104149 25578 raft_consensus.cc:2229] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:27:19.104382 25577 tablet_replica.cc:330] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: stopping tablet replica
I20260811 06:27:19.104645 25578 raft_consensus.cc:2258] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:27:19.106355 25578 ts_tablet_manager.cc:1905] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:27:19.106423 25577 raft_consensus.cc:2229] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:27:19.106911 25577 raft_consensus.cc:2258] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:27:19.107542 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.083181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43990 (local address 127.7.4.129:39133)
0811 06:27:19.083442 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.083451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.083473 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:27:19.083543 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.083550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.083646 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.083752 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:19.086016 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.087588 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.090338 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.091229 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.091387 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.095679 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:19.095694 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:19.095710 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:19.095754 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:19.102049 (+ 6295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.103132 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.103145 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.103151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.103395 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.103924 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.103932 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.103939 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.104659 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:19.105261 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.105826 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.107151 (+ 1325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 06:27:19.108281 25579 ts_tablet_manager.cc:1918] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:27:19.108343 25224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:19.108469 25579 log.cc:1196] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:19.109113 25579 ts_tablet_manager.cc:1939] T 78b8c5a812d245f4ac06d3fe10b50bff P 010474aa2a5045da87d84a24c25251a8: Deleting consensus metadata
I20260811 06:27:19.111544 25001 catalog_manager.cc:4841] TS 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): tablet 78b8c5a812d245f4ac06d3fe10b50bff (table test-table [id=d1ba28239dce4ddfb95c58f3a968d9b9]) successfully deleted
I20260811 06:27:19.113744 25577 ts_tablet_manager.cc:1905] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:27:19.121035 25577 ts_tablet_manager.cc:1918] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:27:19.121205 25577 log.cc:1196] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:19.121742 25577 ts_tablet_manager.cc:1939] T 78b8c5a812d245f4ac06d3fe10b50bff P 3f4d2c7d0c90481f9c5d88dc199f6c16: Deleting consensus metadata
I20260811 06:27:19.123719 24999 catalog_manager.cc:4841] TS 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): tablet 78b8c5a812d245f4ac06d3fe10b50bff (table test-table [id=d1ba28239dce4ddfb95c58f3a968d9b9]) successfully deleted
I20260811 06:27:19.204002 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 25062
W20260811 06:27:19.227870 25000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.4.129:39133 (error 108)
I20260811 06:27:19.228089 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 25233
I20260811 06:27:19.230695 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.229962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44000)
0811 06:27:19.230256 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.230393 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260811 06:27:19.231000 25000 catalog_manager.cc:4584] TS 4cb5fe222a954fb99a0d5c81a27ec24b (127.7.4.129:39133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78b8c5a812d245f4ac06d3fe10b50bff: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
I20260811 06:27:19.242988 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.229044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.7.4.130:34449)
0811 06:27:19.229237 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.229243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.229339 (+ 96us) server_negotiation.cc:407] Connection header received
0811 06:27:19.229500 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.229506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.229643 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.229839 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:19.231511 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.232324 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.233642 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.234108 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.234243 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.236963 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:19.236977 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:19.236991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:19.237021 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:19.240108 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.240834 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.240849 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.240853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.240978 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.241354 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.241360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.241363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.241756 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:19.242094 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.242407 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.242647 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:27:19.243757 25375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:19.284862 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.284417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44004)
0811 06:27:19.284618 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.284709 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 06:27:19.285877 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.285501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44018)
0811 06:27:19.285666 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.285757 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:27:19.346340 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.345827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44024)
0811 06:27:19.346081 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.346167 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 06:27:19.347707 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.347358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44034)
0811 06:27:19.347512 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.347590 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 06:27:19.361819 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 25233
I20260811 06:27:19.385664 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 25385
I20260811 06:27:19.401166 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.386530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.7.4.131:40161)
0811 06:27:19.386738 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:19.386743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:19.386766 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:27:19.386857 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:19.386862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:19.386949 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:19.387051 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:19.388458 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.389603 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.390926 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:19.391585 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:19.391701 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:19.394290 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:19.394312 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:19.394337 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:19.394381 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:19.398051 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.398842 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.398851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.398855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.399001 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:19.399471 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:19.399479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:19.399484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:19.399902 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:19.400214 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:19.400551 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 06:27:19.400816 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:27:19.401942 25524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:19.428335 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.427868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44038)
0811 06:27:19.428069 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.428164 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 06:27:19.429508 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.429106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44050)
0811 06:27:19.429313 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.429399 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 06:27:19.493260 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 25385
I20260811 06:27:19.516286 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 24978
I20260811 06:27:19.517453 25055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:19.638794 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 24978
I20260811 06:27:19.638895 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:19.637973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39133 (local address 127.0.0.1:44062)
0811 06:27:19.638210 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:19.638328 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.129:39133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:27:39.666356 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--webserver_interface=127.7.4.190
--webserver_port=40765
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:45487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:40.002933 25586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:40.003193 25586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:40.003326 25586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:40.010396 25586 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:27:40.010475 25586 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:27:40.010560 25586 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:27:40.010654 25586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:27:40.010738 25586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:40.010851 25586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:40.010923 25586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:27:40.011001 25586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:27:40.020438 25586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:45487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=40765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:40.021553 25586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:40.022811 25586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:40.034401 25596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:40.034449 25594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:40.034493 25593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:40.035190 25586 server_base.cc:1034] running on GCE node
I20260811 06:27:40.035972 25586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:40.037289 25586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:40.038544 25586 hybrid_clock.cc:648] HybridClock initialized: now 1786429660038478 us; error 42 us; skew 500 ppm
I20260811 06:27:40.038868 25586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:40 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429660, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:40.045849 25586 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:27:40.047395 25586 webserver.cc:460] Webserver started at http://127.7.4.190:40765/ using document root <none> and password file <none>
I20260811 06:27:40.048120 25586 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:40.048239 25586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:40.053663 25586 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:27:40.056377 25606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:40.057655 25586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:27:40.057843 25586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "5478333bcb9c48ec8bee9d121d5ae088"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "492a719dcf839c4507385287bb47dc9e"
server_key_iv: "4289f58ebeabf0007e5a114d049436b0"
server_key_version: "encryptionkey@0"
I20260811 06:27:40.058799 25586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:40.086573 25586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:40.091068 25586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:40.091389 25586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:40.109753 25586 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:45487
I20260811 06:27:40.109833 25671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:45487 every 8 connection(s)
I20260811 06:27:40.111181 25586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:27:40.117828 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25586
I20260811 06:27:40.118139 25672 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:27:40.118918 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:39133
--local_ip_for_outbound_sockets=127.7.4.129
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=34863
--webserver_interface=127.7.4.129
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:27:40.123145 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap starting.
I20260811 06:27:40.145891 25672 log.cc:824] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:40.161566 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:27:40.162103 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Bootstrap complete.
I20260811 06:27:40.171558 25672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:40.172447 25672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Initialized, Role: FOLLOWER
I20260811 06:27:40.173096 25672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:40.173318 25672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:27:40.173435 25672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:27:40.173612 25672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:27:40.177438 25672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:40.177898 25672 leader_election.cc:302] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5478333bcb9c48ec8bee9d121d5ae088; no voters:
I20260811 06:27:40.178531 25672 leader_election.cc:288] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:27:40.178666 25676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:27:40.180027 25676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [term 3 LEADER]: Becoming Leader. State: Replica: 5478333bcb9c48ec8bee9d121d5ae088, State: Running, Role: LEADER
I20260811 06:27:40.180550 25676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } }
I20260811 06:27:40.180780 25672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:27:40.182914 25678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5478333bcb9c48ec8bee9d121d5ae088. Latest consensus state: current_term: 3 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:27:40.183272 25678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:40.183528 25677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5478333bcb9c48ec8bee9d121d5ae088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5478333bcb9c48ec8bee9d121d5ae088" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 45487 } } }
I20260811 06:27:40.184193 25677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:40.192550 25685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:27:40.207177 25685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:27:40.208762 25685 catalog_manager.cc:1259] Loaded cluster ID: b4999faa55924498a86ac2789ba0b8b1
I20260811 06:27:40.208876 25685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:27:40.212824 25685 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:27:40.214447 25685 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Loaded TSK: 0
I20260811 06:27:40.214591 25685 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5478333bcb9c48ec8bee9d121d5ae088: Loaded TSK: 1
F20260811 06:27:40.216125 25685 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f61dcb399ad google::LogMessage::Fail() at ??:0
@ 0x7f61dcb3c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f61dcb394de google::LogMessage::Flush() at ??:0
@ 0x7f61dcb3aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f61e089d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f61e088c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f61f256c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f61f256b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f61f25f72a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f61f25f70c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f61f26415e9 std::function<>::operator()() at ??:0
@ 0x7f61f256fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f61f256dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f61f25f5610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f61f25f5439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61df785345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f61df78891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f61df788799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f61df763eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f61e2a3d6db start_thread at ??:0
@ 0x7f61db9d361f clone at ??:0
W20260811 06:27:40.504807 25674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:40.505121 25674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:40.505297 25674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:40.512262 25674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:40.512400 25674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:40.512595 25674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:27:40.521844 25674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:39133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=34863
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:40.523052 25674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:40.524334 25674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:40.536893 25726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:40.537009 25727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:40.537463 25729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:40.537932 25674 server_base.cc:1034] running on GCE node
I20260811 06:27:40.538558 25674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:40.539553 25674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:40.540758 25674 hybrid_clock.cc:648] HybridClock initialized: now 1786429660540706 us; error 42 us; skew 500 ppm
I20260811 06:27:40.541069 25674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:40 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429660, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:40.544102 25674 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:27:40.545482 25674 webserver.cc:460] Webserver started at http://127.7.4.129:34863/ using document root <none> and password file <none>
I20260811 06:27:40.546173 25674 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:40.546320 25674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:40.551582 25674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:27:40.554307 25736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:40.555567 25674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:27:40.555738 25674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "4cb5fe222a954fb99a0d5c81a27ec24b"
format_stamp: "Formatted at 2026-08-11 06:26:54 on dist-test-slave-07c2"
server_key: "8931e35c759180f6b50feb464e93c057"
server_key_iv: "a10f065c9255274d828f05e9c8127fcf"
server_key_version: "encryptionkey@0"
I20260811 06:27:40.556689 25674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:40.589648 25674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:40.593529 25674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:40.593820 25674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:40.594798 25674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:40.596601 25743 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 06:27:40.602013 25674 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 06:27:40.602177 25674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 06:27:40.602308 25674 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 06:27:40.605237 25674 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 06:27:40.605343 25674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260811 06:27:40.605774 25743 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Bootstrap starting.
I20260811 06:27:40.613929 25743 log.cc:824] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:40.629338 25743 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:27:40.630246 25743 tablet_bootstrap.cc:492] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Bootstrap complete.
I20260811 06:27:40.630873 25743 ts_tablet_manager.cc:1397] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.011s
I20260811 06:27:40.639900 25743 raft_consensus.cc:348] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:40.640955 25743 raft_consensus.cc:729] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb5fe222a954fb99a0d5c81a27ec24b, State: Initialized, Role: FOLLOWER
I20260811 06:27:40.641791 25743 consensus_queue.cc:260] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:40.643606 25743 ts_tablet_manager.cc:1428] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260811 06:27:40.649750 25674 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39133
I20260811 06:27:40.649834 25868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39133 every 8 connection(s)
I20260811 06:27:40.650888 25674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:27:40.652334 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25674
I20260811 06:27:40.653198 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:34449
--local_ip_for_outbound_sockets=127.7.4.130
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=43183
--webserver_interface=127.7.4.130
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:27:40.656208 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:40.653908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:44175)
0811 06:27:40.654845 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:40.655097 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":387,"threads_started":1}
W20260811 06:27:40.656906 25870 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45487 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45487: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
I20260811 06:27:40.657846 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:40.657418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:55649)
0811 06:27:40.657606 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:40.657706 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:27:40.658834 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:40.658383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:41741)
0811 06:27:40.658567 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:40.658674 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:27:40.659096 25870 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:27:40.984478 25873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:40.984769 25873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:40.984894 25873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:40.992051 25873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:40.992146 25873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:40.992302 25873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:27:41.001431 25873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:34449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=43183
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:41.002667 25873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:41.003971 25873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:41.016974 25878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:41.016943 25879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:41.017437 25882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:41.017864 25873 server_base.cc:1034] running on GCE node
I20260811 06:27:41.018410 25873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:41.019336 25873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:41.020555 25873 hybrid_clock.cc:648] HybridClock initialized: now 1786429661020512 us; error 40 us; skew 500 ppm
I20260811 06:27:41.020869 25873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:41 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:41.024101 25873 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:27:41.025444 25873 webserver.cc:460] Webserver started at http://127.7.4.130:43183/ using document root <none> and password file <none>
I20260811 06:27:41.026154 25873 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:41.026314 25873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:41.031495 25873 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:27:41.034214 25889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.035452 25873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:27:41.035617 25873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "010474aa2a5045da87d84a24c25251a8"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "514fcef6ae7d1d8b6ebcb0be796a8133"
server_key_iv: "26e155907b8b727ba6a1df7724671e28"
server_key_version: "encryptionkey@0"
I20260811 06:27:41.036576 25873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:41.053705 25873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:41.057946 25873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:41.058228 25873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:41.059182 25873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:41.060536 25873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:41.060676 25873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.060803 25873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:41.060890 25873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.095053 25873 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34449
I20260811 06:27:41.095147 26019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34449 every 8 connection(s)
I20260811 06:27:41.096472 25873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:27:41.101121 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.099389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:51381)
0811 06:27:41.100089 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.100289 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":279,"threads_started":1}
W20260811 06:27:41.101805 26020 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45487 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45487: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
I20260811 06:27:41.102551 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.102260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:46717)
0811 06:27:41.102396 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.102458 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:27:41.103399 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.103067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:49145)
0811 06:27:41.103178 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.103246 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 06:27:41.103641 26020 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:27:41.104274 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 25873
I20260811 06:27:41.104914 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:40161
--local_ip_for_outbound_sockets=127.7.4.131
--tserver_master_addrs=127.7.4.190:45487
--webserver_port=44951
--webserver_interface=127.7.4.131
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:41.436334 26023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:41.436619 26023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:41.436709 26023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:41.443745 26023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:41.443864 26023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:41.443972 26023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:27:41.453161 26023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:40161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=44951
--tserver_master_addrs=127.7.4.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:41.454411 26023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:41.455713 26023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:41.468730 26028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:41.468775 26029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:41.468736 26031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:41.469772 26023 server_base.cc:1034] running on GCE node
I20260811 06:27:41.470345 26023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:41.471158 26023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:41.472354 26023 hybrid_clock.cc:648] HybridClock initialized: now 1786429661472289 us; error 76 us; skew 500 ppm
I20260811 06:27:41.472618 26023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:41 dist-test-slave-07c2 krb5kdc[24349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429661, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:41.475890 26023 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:27:41.477269 26023 webserver.cc:460] Webserver started at http://127.7.4.131:44951/ using document root <none> and password file <none>
I20260811 06:27:41.477880 26023 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:41.477990 26023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:41.482924 26023 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:27:41.485780 26038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.487025 26023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:27:41.487241 26023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16"
format_stamp: "Formatted at 2026-08-11 06:26:55 on dist-test-slave-07c2"
server_key: "0cadf7b4d6245a99f23ee53871ed4156"
server_key_iv: "caccc1b189662d38c01d3f7aeccb7eec"
server_key_version: "encryptionkey@0"
I20260811 06:27:41.488270 26023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:41.509351 26023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:41.513563 26023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:41.513850 26023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:41.514887 26023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:41.516215 26023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:41.516311 26023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.516466 26023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:41.516559 26023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:41.550976 26023 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:40161
I20260811 06:27:41.551074 26166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:40161 every 8 connection(s)
I20260811 06:27:41.552449 26023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:27:41.555375 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26023
I20260811 06:27:41.557662 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.555477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:39623)
0811 06:27:41.556368 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.556670 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":363,"threads_started":1}
W20260811 06:27:41.558315 26167 heartbeater.cc:644] Failed to heartbeat to 127.7.4.190:45487 (0 consecutive failures): Network error: Failed to ping master at 127.7.4.190:45487: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
I20260811 06:27:41.559124 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.558792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:49073)
0811 06:27:41.558952 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.559015 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:27:41.559999 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.559684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:45321)
0811 06:27:41.559813 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.559888 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 06:27:41.560251 26167 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:27:41.660318 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:41.659597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:60731)
0811 06:27:41.660006 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:41.660078 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":196,"threads_started":1}
I20260811 06:27:42.104852 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.104101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:55603)
0811 06:27:42.104527 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:42.104610 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":205,"threads_started":1}
I20260811 06:27:42.168706 26173 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:42.169031 26173 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:42.170527 26173 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161), 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
Aug 11 06:27:42 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429660, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
Aug 11 06:27:42 dist-test-slave-07c2 krb5kdc[24349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429660, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:27:42.192296 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.170778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40161 (local address 127.7.4.129:50077)
0811 06:27:42.171357 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:42.171395 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:42.172133 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:42.175214 (+ 3081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:42.175224 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:42.176198 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:42.177034 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:42.177054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.178878 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.178884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:42.180304 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:42.180317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.181269 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.181275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:42.181509 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:42.182239 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:42.182264 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:42.184680 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:42.188299 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:42.188311 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:42.188325 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:42.188929 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:42.189462 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:42.189466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:42.189468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:42.189724 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:42.190640 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:42.190647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:42.190980 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 06:27:42.191265 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 06:27:42.191795 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":202,"spinlock_wait_cycles":59904,"thread_start_us":599,"threads_started":1}
I20260811 06:27:42.192480 26177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.171248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:49359 (local address 127.7.4.130:34449)
0811 06:27:42.173004 (+ 1756us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:42.173014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:42.173048 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:27:42.174581 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:42.174588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:42.174907 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:42.175259 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:42.177251 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.178729 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.180446 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.181029 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.181223 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:42.185321 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:42.185327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:42.185337 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:42.185370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:42.188895 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:42.189693 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:42.189701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:42.189709 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:42.189915 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:42.190308 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:42.190311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:42.190313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:42.190899 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:42.191086 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:42.191711 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 06:27:42.192079 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606,"thread_start_us":1420,"threads_started":1}
I20260811 06:27:42.192392 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.170783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34449 (local address 127.7.4.129:49359)
0811 06:27:42.171685 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:42.171709 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:42.172141 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:42.175231 (+ 3090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:42.175236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:42.176198 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:42.177035 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:42.177055 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.178909 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.178913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:42.180307 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:42.180319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.181236 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.181247 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:42.181511 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:42.182309 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:42.182338 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:42.185077 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:42.189055 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:42.189060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:42.189063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:42.189543 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:42.190036 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:42.190040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:42.190043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:42.190201 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:42.191028 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:42.191031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:42.191214 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 06:27:42.191425 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 06:27:42.191941 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":334,"spinlock_wait_cycles":25728,"thread_start_us":229,"threads_started":1}
I20260811 06:27:42.192996 26176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.171182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:50077 (local address 127.7.4.131:40161)
0811 06:27:42.172893 (+ 1711us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:42.172903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:42.172934 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:27:42.174415 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:42.174424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:42.174781 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:42.175026 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:42.177236 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.178694 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.180445 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:42.181036 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:42.181239 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:42.185004 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:42.185014 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:42.185047 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:42.185089 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:42.188141 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:42.189084 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:42.189094 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:42.189103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:42.189324 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:42.189864 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:42.189868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:42.189870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:42.190502 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:42.190640 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:42.191572 (+ 932us) server_negotiation.cc:299] Negotiation successful
0811 06:27:42.192097 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1543,"thread_start_us":1435,"threads_started":1}
W20260811 06:27:42.195555 25738 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
W20260811 06:27:42.195776 25740 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:42.195945 25740 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb5fe222a954fb99a0d5c81a27ec24b; no voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16
I20260811 06:27:42.196369 26173 raft_consensus.cc:2726] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:27:42.561580 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.560764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:33893)
0811 06:27:42.561216 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:42.561337 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":210,"threads_started":1}
I20260811 06:27:42.661597 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:42.661171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:41515)
0811 06:27:42.661357 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:42.661449 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:27:43.106577 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:43.105871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:37379)
0811 06:27:43.106342 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:43.106417 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":254,"threads_started":1}
I20260811 06:27:43.563056 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:43.562417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:40545)
0811 06:27:43.562822 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:43.562899 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":216,"threads_started":1}
I20260811 06:27:43.663179 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:43.662370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:60229)
0811 06:27:43.662887 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:43.662981 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":255,"threads_started":1}
I20260811 06:27:43.782635 26184 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:43.782869 26184 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:43.783720 26184 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161), 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
W20260811 06:27:43.784760 25740 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
W20260811 06:27:43.784962 25738 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:43.785133 25738 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb5fe222a954fb99a0d5c81a27ec24b; no voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16
I20260811 06:27:43.785424 26184 raft_consensus.cc:2726] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:27:44.108242 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:44.107568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:44971)
0811 06:27:44.108006 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:44.108083 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":229,"threads_started":1}
I20260811 06:27:44.564584 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:44.563939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:45453)
0811 06:27:44.564353 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:44.564428 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":206,"threads_started":1}
I20260811 06:27:44.664709 26187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:44.664032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:46469)
0811 06:27:44.664464 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:44.664541 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":236,"threads_started":1}
I20260811 06:27:45.109865 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:45.109094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:48241)
0811 06:27:45.109601 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:45.109678 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":288,"threads_started":1}
I20260811 06:27:45.566113 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:45.565429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:50165)
0811 06:27:45.565853 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:45.565933 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":219,"threads_started":1}
I20260811 06:27:45.666508 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:45.665673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:42737)
0811 06:27:45.666202 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:45.666310 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":261,"threads_started":1}
I20260811 06:27:45.683439 26192 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:45.683687 26192 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:45.684361 26192 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161), 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
W20260811 06:27:45.685320 25740 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
W20260811 06:27:45.685458 25738 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:45.685644 25738 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb5fe222a954fb99a0d5c81a27ec24b; no voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16
I20260811 06:27:45.685884 26192 raft_consensus.cc:2726] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:27:46.111474 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:46.110689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:49971)
0811 06:27:46.111168 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:46.111244 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":244,"threads_started":1}
I20260811 06:27:46.567759 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:46.567057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:45975)
0811 06:27:46.567523 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:46.567598 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":270,"threads_started":1}
I20260811 06:27:46.668040 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:46.667358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:38231)
0811 06:27:46.667761 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:46.667859 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":213,"threads_started":1}
I20260811 06:27:47.113211 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:47.112347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:41461)
0811 06:27:47.112868 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:47.112948 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
I20260811 06:27:47.569568 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:47.568779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:45855)
0811 06:27:47.569247 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:47.569383 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260811 06:27:47.669710 26201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:47.668910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:60785)
0811 06:27:47.669435 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:47.669511 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":303,"threads_started":1}
I20260811 06:27:48.115077 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:48.114292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:48995)
0811 06:27:48.114812 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:48.114894 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":236,"threads_started":1}
I20260811 06:27:48.571390 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:48.570514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:39155)
0811 06:27:48.571040 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:48.571122 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":249,"threads_started":1}
I20260811 06:27:48.671197 26204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:48.670549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:53537)
0811 06:27:48.670957 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:48.671033 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":208,"threads_started":1}
I20260811 06:27:49.116669 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:49.115961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:56601)
0811 06:27:49.116396 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:49.116490 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":235,"threads_started":1}
I20260811 06:27:49.382145 26206 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:49.382375 26206 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:49.383070 26206 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161), 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
W20260811 06:27:49.384013 25738 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
W20260811 06:27:49.384231 25740 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:49.384368 25740 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb5fe222a954fb99a0d5c81a27ec24b; no voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16
I20260811 06:27:49.384635 26206 raft_consensus.cc:2726] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:27:49.573577 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:49.572385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:43669)
0811 06:27:49.573306 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:49.573387 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"spinlock_wait_cycles":642944,"thread_start_us":328,"threads_started":1}
I20260811 06:27:49.672925 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:49.672093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:49191)
0811 06:27:49.672625 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:49.672705 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":263,"threads_started":1}
I20260811 06:27:50.118337 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:50.117619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:39407)
0811 06:27:50.118084 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:50.118185 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":227,"threads_started":1}
I20260811 06:27:50.575538 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:50.574733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:40663)
0811 06:27:50.575240 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:50.575378 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":261,"threads_started":1}
I20260811 06:27:50.674646 26211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:50.673875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:60313)
0811 06:27:50.674413 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:50.674490 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":330,"threads_started":1}
I20260811 06:27:51.120010 26212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:51.119253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:45747)
0811 06:27:51.119731 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:51.119824 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":238,"threads_started":1}
I20260811 06:27:51.577576 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:51.576579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:56817)
0811 06:27:51.577203 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:51.577330 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":378,"threads_started":1}
I20260811 06:27:51.676213 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:51.675546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:43013)
0811 06:27:51.675969 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:51.676043 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":222,"threads_started":1}
I20260811 06:27:52.121858 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:52.120987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:43623)
0811 06:27:52.121541 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:52.121638 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260811 06:27:52.579375 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:52.578551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:50223)
0811 06:27:52.579062 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:52.579145 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":261,"threads_started":1}
I20260811 06:27:52.677716 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:52.677066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:55813)
0811 06:27:52.677488 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:52.677565 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260811 06:27:53.123662 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:53.122871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:51651)
0811 06:27:53.123388 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:53.123467 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":232,"threads_started":1}
I20260811 06:27:53.581281 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:53.580309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:33363)
0811 06:27:53.580915 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:53.581017 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":316,"threads_started":1}
I20260811 06:27:53.679207 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:53.678525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:58595)
0811 06:27:53.678960 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:53.679033 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":232,"threads_started":1}
I20260811 06:27:54.125155 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:54.124476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:49243)
0811 06:27:54.124927 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:54.125001 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":210,"threads_started":1}
I20260811 06:27:54.582980 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:54.582299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:50801)
0811 06:27:54.582732 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:54.582810 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260811 06:27:54.680698 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:54.680038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:50361)
0811 06:27:54.680442 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:54.680515 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":224,"threads_started":1}
I20260811 06:27:55.126688 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:55.125988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:36433)
0811 06:27:55.126423 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:55.126501 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260811 06:27:55.526888 26226 raft_consensus.cc:482] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:27:55.527129 26226 raft_consensus.cc:504] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb5fe222a954fb99a0d5c81a27ec24b" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39133 } } peers { permanent_uuid: "3f4d2c7d0c90481f9c5d88dc199f6c16" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40161 } } peers { permanent_uuid: "010474aa2a5045da87d84a24c25251a8" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34449 } }
I20260811 06:27:55.527815 26226 leader_election.cc:288] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161), 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449)
W20260811 06:27:55.528704 25738 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f4d2c7d0c90481f9c5d88dc199f6c16 (127.7.4.131:40161): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
W20260811 06:27:55.528895 25740 leader_election.cc:341] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 010474aa2a5045da87d84a24c25251a8 (127.7.4.130:34449): Not found: Tablet not found: 78b8c5a812d245f4ac06d3fe10b50bff
I20260811 06:27:55.529028 25740 leader_election.cc:302] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb5fe222a954fb99a0d5c81a27ec24b; no voters: 010474aa2a5045da87d84a24c25251a8, 3f4d2c7d0c90481f9c5d88dc199f6c16
I20260811 06:27:55.529265 26226 raft_consensus.cc:2726] T 78b8c5a812d245f4ac06d3fe10b50bff P 4cb5fe222a954fb99a0d5c81a27ec24b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:27:55.584622 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:55.583883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:58881)
0811 06:27:55.584346 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:55.584425 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260811 06:27:55.682761 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:55.681880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.129:44739)
0811 06:27:55.682462 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:55.682559 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":337,"threads_started":1}
I20260811 06:27:56.128428 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:56.127764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.130:38505)
0811 06:27:56.128191 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:56.128276 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260811 06:27:56.558998 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 25674
I20260811 06:27:56.577147 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:56.559978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.7.4.129:39133)
0811 06:27:56.560455 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:56.560462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:56.560488 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:27:56.560548 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:56.560554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:56.560764 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:56.560960 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:56.562380 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.563472 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.564740 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.565265 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.565372 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:56.571986 (+ 6614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:56.572015 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:56.572033 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:56.572089 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:56.574661 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.575340 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.575352 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.575354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.575490 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.575810 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.575813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.575815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.576164 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:56.576401 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:56.576599 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 06:27:56.576792 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":217,"threads_started":1}
I20260811 06:27:56.578372 25863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:56.586406 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:56.585712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:45487 (local address 127.7.4.131:57759)
0811 06:27:56.586154 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:56.586254 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.190:45487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":234,"threads_started":1}
I20260811 06:27:56.666790 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 25674
I20260811 06:27:56.690266 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 25873
I20260811 06:27:56.703760 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:56.691016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.7.4.130:34449)
0811 06:27:56.691581 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:56.691586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:56.691602 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:27:56.691658 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:56.691661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:56.691727 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:56.691817 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:56.693262 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.694067 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.695311 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.695773 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.695911 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:56.698376 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:56.698381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:56.698384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:56.698417 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:56.701091 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.701786 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.701793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.701797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.701923 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.702317 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.702323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.702327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.702587 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:56.702844 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:56.703136 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 06:27:56.703377 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":353,"threads_started":1}
I20260811 06:27:56.704416 26015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:56.790241 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 25873
I20260811 06:27:56.812745 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26023
I20260811 06:27:56.826583 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:56.813717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.7.4.131:40161)
0811 06:27:56.814301 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:56.814306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:56.814322 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:27:56.814378 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:56.814381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:56.814447 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:56.814569 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:56.816027 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.816810 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.818085 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:56.818510 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:56.818656 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:56.821207 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:56.821211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:56.821215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:56.821247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:56.823894 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.824548 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.824556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.824559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.824714 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:56.825108 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:56.825114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:56.825117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:56.825368 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:56.825637 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:56.825952 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 06:27:56.826197 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":289,"threads_started":1}
I20260811 06:27:56.827337 26162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:27:56.910420 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26023
2026-08-11T06:27:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64405 ms)
[----------] 30 tests from SecurityITest (302929 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:27:57 dist-test-slave-07c2 krb5kdc[26242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:27:57 dist-test-slave-07c2 krb5kdc[26242](info): set up 2 sockets
Aug 11 06:27:57 dist-test-slave-07c2 krb5kdc[26242](info): commencing operation
krb5kdc: starting...
W20260811 06:27:58.092514 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:27:58 dist-test-slave-07c2 krb5kdc[26242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:27:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:27:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:27:58.387081 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44193
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:44193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:58.721282 26256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:58.721585 26256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:58.721705 26256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:58.728937 26256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:27:58.729063 26256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:58.729159 26256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:58.729250 26256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:27:58.729326 26256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:27:58.738803 26256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:44193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:58.740042 26256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:58.741259 26256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:58.753311 26262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:58.753360 26265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:58.753360 26263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:58.753988 26256 server_base.cc:1034] running on GCE node
I20260811 06:27:58.754761 26256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:58.756136 26256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:58.757364 26256 hybrid_clock.cc:648] HybridClock initialized: now 1786429678757325 us; error 45 us; skew 500 ppm
I20260811 06:27:58.757689 26256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:58 dist-test-slave-07c2 krb5kdc[26242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:58.760711 26256 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:27:58.762065 26256 webserver.cc:460] Webserver started at http://127.7.4.190:40531/ using document root <none> and password file <none>
I20260811 06:27:58.762746 26256 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:58.762897 26256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:58.763182 26256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:27:58.766000 26256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8"
format_stamp: "Formatted at 2026-08-11 06:27:58 on dist-test-slave-07c2"
server_key: "c5439d1cd9d24025796730a4e861f1c6"
server_key_iv: "869f62669a62db7e1332cbc93f8affde"
server_key_version: "encryptionkey@0"
I20260811 06:27:58.766747 26256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8"
format_stamp: "Formatted at 2026-08-11 06:27:58 on dist-test-slave-07c2"
server_key: "c5439d1cd9d24025796730a4e861f1c6"
server_key_iv: "869f62669a62db7e1332cbc93f8affde"
server_key_version: "encryptionkey@0"
I20260811 06:27:58.771667 26256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 06:27:58.774785 26277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:58.776033 26256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:27:58.776206 26256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8"
format_stamp: "Formatted at 2026-08-11 06:27:58 on dist-test-slave-07c2"
server_key: "c5439d1cd9d24025796730a4e861f1c6"
server_key_iv: "869f62669a62db7e1332cbc93f8affde"
server_key_version: "encryptionkey@0"
I20260811 06:27:58.776400 26256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:58.796907 26256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:58.800551 26256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:58.800825 26256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:58.819234 26256 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:44193
I20260811 06:27:58.819278 26334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:44193 every 8 connection(s)
I20260811 06:27:58.820525 26256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:27:58.823645 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26256
I20260811 06:27:58.823863 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:27:58.824173 26335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:27:58.824575 7186 external_mini_cluster.cc:1428] Setting key ef69b736f3f86a0f534d1a8ec24bdbec
I20260811 06:27:58.838039 26335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: Bootstrap starting.
Aug 11 06:27:58 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:58.841426 26335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: Neither blocks nor log segments found. Creating new log.
I20260811 06:27:58.842805 26335 log.cc:824] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:27:58.845826 26335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: No bootstrap required, opened a new log
I20260811 06:27:58.848551 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:58.826435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55328 (local address 127.7.4.190:44193)
0811 06:27:58.827307 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:58.827326 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:58.827373 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:27:58.829198 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:58.829229 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:58.829713 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:58.830127 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:58.831486 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:58.833425 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:58.834937 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:58.836015 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:58.836193 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:58.840511 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:58.840522 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:58.840541 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:58.840584 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:58.844309 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:58.844971 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:58.844979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:58.844986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:58.845196 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:58.845534 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:58.845539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:58.845542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:58.846204 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:58.846521 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:58.846834 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:27:58.847232 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":372,"threads_started":1}
I20260811 06:27:58.854957 26335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } }
I20260811 06:27:58.855387 26335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:27:58.855474 26335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eac2d86a7cf417eb228a2e4c0e8ddd8, State: Initialized, Role: FOLLOWER
I20260811 06:27:58.855935 26335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } }
I20260811 06:27:58.856199 26335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:27:58.856382 26335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:27:58.856575 26335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:27:58.858649 26335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } }
I20260811 06:27:58.859109 26335 leader_election.cc:302] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9eac2d86a7cf417eb228a2e4c0e8ddd8; no voters:
I20260811 06:27:58.859696 26335 leader_election.cc:288] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:27:58.859862 26341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:27:58.861078 26341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9eac2d86a7cf417eb228a2e4c0e8ddd8, State: Running, Role: LEADER
I20260811 06:27:58.861567 26341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } }
I20260811 06:27:58.861724 26335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:27:58.864522 26343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9eac2d86a7cf417eb228a2e4c0e8ddd8. Latest consensus state: current_term: 1 leader_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } } }
I20260811 06:27:58.864948 26343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:58.864486 26342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9eac2d86a7cf417eb228a2e4c0e8ddd8" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 44193 } } }
I20260811 06:27:58.865291 26342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:27:58.867147 26352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:27:58.871569 26352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:27:58.879822 26352 catalog_manager.cc:1347] Generated new cluster ID: e3c0a88bd02f42d5a8c5a9684bae4696
I20260811 06:27:58.879927 26352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:27:58.891820 26352 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:27:58.893030 26352 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:27:58.907984 26352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: Generated new TSK 0
I20260811 06:27:58.908690 26352 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:27:58.991146 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:59.330336 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:59.330600 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:59.330696 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:59.337904 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:59.338021 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:59.338203 26366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:27:59.347775 26366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:59.348956 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:59.350229 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:59.363083 26373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:59.363129 26372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:59.363200 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:59.364706 26366 server_base.cc:1034] running on GCE node
I20260811 06:27:59.365322 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:59.366237 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:59.367434 26366 hybrid_clock.cc:648] HybridClock initialized: now 1786429679367374 us; error 51 us; skew 500 ppm
I20260811 06:27:59.367758 26366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:59 dist-test-slave-07c2 krb5kdc[26242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429679, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:59.370824 26366 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:27:59.372150 26366 webserver.cc:460] Webserver started at http://127.7.4.129:43371/ using document root <none> and password file <none>
I20260811 06:27:59.372939 26366 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:59.373044 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:59.373327 26366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:27:59.376271 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "90f097e59505492fa0d1f70114cc028a"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "d843694c20457fd4b2fe6a3c508f2f51"
server_key_iv: "c491698384e14dc877f7efcf18aa3792"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.376999 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "90f097e59505492fa0d1f70114cc028a"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "d843694c20457fd4b2fe6a3c508f2f51"
server_key_iv: "c491698384e14dc877f7efcf18aa3792"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.381939 26366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:27:59.385105 26383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.386370 26366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:27:59.386533 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "90f097e59505492fa0d1f70114cc028a"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "d843694c20457fd4b2fe6a3c508f2f51"
server_key_iv: "c491698384e14dc877f7efcf18aa3792"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.386741 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:59.404477 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:59.408313 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:59.408604 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:59.409489 26366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:59.410715 26366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:59.410825 26366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.410964 26366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:59.411029 26366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.444667 26366 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:38497
I20260811 06:27:59.444747 26515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:38497 every 8 connection(s)
I20260811 06:27:59.446221 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:27:59.448873 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26366
I20260811 06:27:59.449107 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:27:59.449882 7186 external_mini_cluster.cc:1428] Setting key f26943660a6f55fe98d440167aa5057b
Aug 11 06:27:59 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429679, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:59.467986 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:59.448950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:52199 (local address 127.7.4.190:44193)
0811 06:27:59.449539 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:59.449545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:59.450567 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 06:27:59.452540 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:59.452549 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:59.452638 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:59.452757 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:59.454110 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.455272 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.456806 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.457493 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.457640 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:59.461058 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:59.461067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:59.461075 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:59.461119 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:59.464839 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:59.465513 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:59.465521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:59.465527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:59.465692 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:59.466061 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:59.466068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:59.466074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:59.466467 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:59.466590 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:59.467533 (+ 943us) server_negotiation.cc:299] Negotiation successful
0811 06:27:59.467725 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":311,"threads_started":1}
I20260811 06:27:59.469445 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:59.449532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44193 (local address 127.7.4.129:52199)
0811 06:27:59.450399 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:59.450452 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:59.452273 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:59.452938 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:59.452945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:59.453441 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:59.453951 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:59.453967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.455430 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.455434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:59.456657 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:59.456669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.457719 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.457724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:59.457935 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:59.458621 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:59.458647 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:59.460840 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:59.464968 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:59.464974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:59.464985 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:59.465400 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:59.465801 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:59.465804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:59.465806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:59.465975 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:59.466606 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:59.466613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:59.467065 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 06:27:59.467480 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 06:27:59.468075 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":362,"threads_started":1}
I20260811 06:27:59.470985 26516 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44193
I20260811 06:27:59.471312 26516 heartbeater.cc:464] Registering TS with master...
I20260811 06:27:59.472021 26516 heartbeater.cc:511] Master 127.7.4.190:44193 requested a full tablet report, sending...
I20260811 06:27:59.474014 26295 ts_manager.cc:194] Registered new tserver with Master: 90f097e59505492fa0d1f70114cc028a (127.7.4.129:38497)
I20260811 06:27:59.475490 26295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:52199
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:27:59.520582 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:27:59.856441 26525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:27:59.856698 26525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:27:59.856791 26525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:27:59.863907 26525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:27:59.863998 26525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:27:59.864107 26525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:27:59.873377 26525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:27:59.874445 26525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:27:59.875633 26525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:27:59.888156 26533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:59.888274 26530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:27:59.888238 26531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:27:59.889053 26525 server_base.cc:1034] running on GCE node
I20260811 06:27:59.889602 26525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:27:59.890503 26525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:27:59.891717 26525 hybrid_clock.cc:648] HybridClock initialized: now 1786429679891668 us; error 48 us; skew 500 ppm
I20260811 06:27:59.891979 26525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:27:59 dist-test-slave-07c2 krb5kdc[26242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429679, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:27:59.895015 26525 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:27:59.896512 26525 webserver.cc:460] Webserver started at http://127.7.4.130:44253/ using document root <none> and password file <none>
I20260811 06:27:59.897146 26525 fs_manager.cc:359] Metadata directory not provided
I20260811 06:27:59.897274 26525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:27:59.897595 26525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:27:59.900372 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "2b3f918344634dafa113490ea1dd02f9"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "95b4cb23c5d00c4632f633a773de5cb3"
server_key_iv: "4f2759f84065cb870eb612af1f095f69"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.901099 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "2b3f918344634dafa113490ea1dd02f9"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "95b4cb23c5d00c4632f633a773de5cb3"
server_key_iv: "4f2759f84065cb870eb612af1f095f69"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.905756 26525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 06:27:59.908995 26544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.910279 26525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:27:59.910444 26525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "2b3f918344634dafa113490ea1dd02f9"
format_stamp: "Formatted at 2026-08-11 06:27:59 on dist-test-slave-07c2"
server_key: "95b4cb23c5d00c4632f633a773de5cb3"
server_key_iv: "4f2759f84065cb870eb612af1f095f69"
server_key_version: "encryptionkey@0"
I20260811 06:27:59.910621 26525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:27:59.928957 26525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:27:59.932688 26525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:27:59.932971 26525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:27:59.933888 26525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:27:59.935230 26525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:27:59.935359 26525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.935501 26525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:27:59.935552 26525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:27:59.968669 26525 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:41569
I20260811 06:27:59.968713 26679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:41569 every 8 connection(s)
I20260811 06:27:59.969915 26525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:27:59.979017 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26525
I20260811 06:27:59.979204 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:27:59.979650 7186 external_mini_cluster.cc:1428] Setting key bf9ee109effa266c18dc198d59f47699
Aug 11 06:27:59 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429679, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:27:59.989759 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:59.972343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:32883 (local address 127.7.4.190:44193)
0811 06:27:59.972828 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:27:59.972837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:27:59.973625 (+ 788us) server_negotiation.cc:407] Connection header received
0811 06:27:59.975482 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:27:59.975488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:27:59.975578 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:27:59.975711 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:27:59.977176 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.977906 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.979528 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.980074 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.980215 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:59.984105 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:27:59.984109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:27:59.984112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:27:59.984142 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:27:59.986711 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:59.987454 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:59.987458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:59.987460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:59.987585 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:27:59.988036 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:27:59.988040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:27:59.988042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:27:59.988294 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:27:59.988451 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:27:59.989127 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 06:27:59.989354 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":237,"threads_started":1}
I20260811 06:27:59.990710 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:27:59.972733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44193 (local address 127.7.4.130:32883)
0811 06:27:59.973465 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:27:59.973504 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:27:59.975179 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:27:59.975950 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:27:59.975958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:27:59.976483 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:27:59.977004 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:59.977020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.978062 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.978066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:59.979391 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:27:59.979404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:27:59.980338 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:27:59.980347 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:27:59.980594 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:27:59.981333 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:27:59.981363 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:27:59.983899 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:27:59.986853 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:59.986862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:59.986875 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:59.987324 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:59.987722 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:27:59.987727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:27:59.987731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:27:59.987912 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:27:59.988424 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:27:59.988431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:27:59.988766 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 06:27:59.989029 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 06:27:59.989509 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":280,"threads_started":1}
I20260811 06:27:59.992238 26680 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44193
I20260811 06:27:59.992527 26680 heartbeater.cc:464] Registering TS with master...
I20260811 06:27:59.993222 26680 heartbeater.cc:511] Master 127.7.4.190:44193 requested a full tablet report, sending...
I20260811 06:27:59.994590 26295 ts_manager.cc:194] Registered new tserver with Master: 2b3f918344634dafa113490ea1dd02f9 (127.7.4.130:41569)
I20260811 06:27:59.995429 26295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:32883
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:28:00.048280 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:00.382071 26686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:00.382337 26686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:00.382431 26686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:00.389853 26686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:00.389954 26686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:00.390110 26686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:00.399433 26686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:44193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:00.400506 26686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:00.401676 26686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:00.414283 26694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:00.414309 26696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:00.414629 26693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:00.415443 26686 server_base.cc:1034] running on GCE node
I20260811 06:28:00.416015 26686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:00.416831 26686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:00.418023 26686 hybrid_clock.cc:648] HybridClock initialized: now 1786429680417981 us; error 34 us; skew 500 ppm
I20260811 06:28:00.418313 26686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:00 dist-test-slave-07c2 krb5kdc[26242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429680, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:00.421299 26686 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:28:00.422691 26686 webserver.cc:460] Webserver started at http://127.7.4.131:34579/ using document root <none> and password file <none>
I20260811 06:28:00.423363 26686 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:00.423498 26686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:00.423806 26686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:00.426627 26686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "ea26b7b325ec47fa95723cb2932287ce"
format_stamp: "Formatted at 2026-08-11 06:28:00 on dist-test-slave-07c2"
server_key: "f3fbe25a85675479c524a8961d79c466"
server_key_iv: "d4cc27df58eb463250024709f16846d7"
server_key_version: "encryptionkey@0"
I20260811 06:28:00.427479 26686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "ea26b7b325ec47fa95723cb2932287ce"
format_stamp: "Formatted at 2026-08-11 06:28:00 on dist-test-slave-07c2"
server_key: "f3fbe25a85675479c524a8961d79c466"
server_key_iv: "d4cc27df58eb463250024709f16846d7"
server_key_version: "encryptionkey@0"
I20260811 06:28:00.432231 26686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:28:00.435431 26705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:00.436651 26686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:00.436821 26686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "ea26b7b325ec47fa95723cb2932287ce"
format_stamp: "Formatted at 2026-08-11 06:28:00 on dist-test-slave-07c2"
server_key: "f3fbe25a85675479c524a8961d79c466"
server_key_iv: "d4cc27df58eb463250024709f16846d7"
server_key_version: "encryptionkey@0"
I20260811 06:28:00.437011 26686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:00.478251 26516 heartbeater.cc:503] Master 127.7.4.190:44193 was elected leader, sending a full tablet report...
I20260811 06:28:00.488283 26686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:00.492537 26686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:00.492833 26686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:00.493752 26686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:00.495059 26686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:00.495163 26686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:00.495319 26686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:00.495391 26686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 06:28:00.528344 26686 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:34987
I20260811 06:28:00.528400 26844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:34987 every 8 connection(s)
I20260811 06:28:00.529616 26686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:00.537104 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26686
I20260811 06:28:00.537315 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:00.537766 7186 external_mini_cluster.cc:1428] Setting key d9d1c870af4d7e53ef0e82bc3753ee4c
Aug 11 06:28:00 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429680, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:00.550009 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.532191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:34227 (local address 127.7.4.190:44193)
0811 06:28:00.532640 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.532646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.533640 (+ 994us) server_negotiation.cc:407] Connection header received
0811 06:28:00.535493 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.535498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.535568 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.535675 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:00.537293 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.538127 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.540383 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.540990 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.541101 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.544480 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:00.544485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:00.544488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:00.544527 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:00.546953 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.547728 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.547732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.547735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.547851 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.548274 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.548279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.548288 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.548563 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:00.548730 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.549372 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.549592 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":191,"threads_started":1}
I20260811 06:28:00.551067 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.532803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:44193 (local address 127.7.4.131:34227)
0811 06:28:00.533494 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.533532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.535186 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.535875 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.535882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.536368 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:00.537082 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.537112 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.538295 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.538299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.540214 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.540226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.541204 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.541211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.541450 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.542075 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:00.542095 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:00.544275 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:00.547096 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:00.547105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:00.547118 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:00.547594 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:00.547961 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:00.547966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:00.547969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:00.548155 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:00.548699 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:00.548707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:00.549006 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.549259 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.549753 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":293,"threads_started":1}
I20260811 06:28:00.552585 26845 heartbeater.cc:347] Connected to a master server at 127.7.4.190:44193
I20260811 06:28:00.552932 26845 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:00.553654 26845 heartbeater.cc:511] Master 127.7.4.190:44193 requested a full tablet report, sending...
I20260811 06:28:00.555013 26294 ts_manager.cc:194] Registered new tserver with Master: ea26b7b325ec47fa95723cb2932287ce (127.7.4.131:34987)
I20260811 06:28:00.556066 26294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:34227
I20260811 06:28:00.565291 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:00.582461 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.570003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.7.4.190:44193)
0811 06:28:00.570215 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.570223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.570633 (+ 410us) server_negotiation.cc:407] Connection header received
0811 06:28:00.570770 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.570776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.570848 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.570948 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:00.572529 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.573209 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.574462 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.574866 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.574958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.577159 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:00.577163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:00.577167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:00.577196 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:00.579936 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.580586 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.580590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.580592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.580701 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.581041 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.581044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.581046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.581346 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:00.581524 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.582001 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.582154 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:28:00.586316 26294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:00.593248 26294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:00.618652 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.604972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.7.4.130:41569)
0811 06:28:00.606244 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.606257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.606924 (+ 667us) server_negotiation.cc:407] Connection header received
0811 06:28:00.607415 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.607423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.607671 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.607840 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:00.608891 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.610980 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.614735 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.616744 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.616860 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.617078 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.617971 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.618223 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":908,"threads_started":1}
I20260811 06:28:00.620117 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.604908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:41569 (local address 127.0.0.1:60902)
0811 06:28:00.606779 (+ 1871us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.606814 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.607078 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.608163 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.608170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.608200 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:00.608658 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.608679 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.611232 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.611240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.614519 (+ 3279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.614540 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.617320 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.617329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.617628 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.617695 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.619257 (+ 1562us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.619564 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"thread_start_us":1499,"threads_started":1}
I20260811 06:28:00.624182 26605 tablet_service.cc:1467] Processing CreateTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 (DEFAULT_TABLE table=test-table [id=40fcb982b1524e15a3c214060b4ebc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:00.625628 26605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa09ba0345a74b12b820f0e2638e06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:00.627087 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.605062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:38497 (local address 127.0.0.1:60530)
0811 06:28:00.607461 (+ 2399us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.607479 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.607579 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.619089 (+ 11510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.619097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.619135 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:00.619657 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.619673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.622139 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.622144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.624455 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.624474 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.626271 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.626277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.626447 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.626480 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.626568 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.626700 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2289,"mutex_wait_us":3,"thread_start_us":2107,"threads_started":1}
I20260811 06:28:00.627158 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.605625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.7.4.129:38497)
0811 06:28:00.618129 (+ 12504us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.618139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.618168 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:28:00.618281 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.618292 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.618568 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.619946 (+ 1378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:00.620302 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.621950 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.624635 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.626085 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.626225 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.626448 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.626659 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.626832 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12308,"thread_start_us":13138,"threads_started":1}
I20260811 06:28:00.629335 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.607669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:34987 (local address 127.0.0.1:34570)
0811 06:28:00.620568 (+ 12899us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.620587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.620674 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.622024 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.622029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.622054 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:00.622371 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.622385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.624699 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.624705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.625985 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.625997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.628563 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.628577 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.628778 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.628810 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.628902 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.629027 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12786,"mutex_wait_us":49,"thread_start_us":13414,"threads_started":1}
I20260811 06:28:00.630203 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.608165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.7.4.131:34987)
0811 06:28:00.621493 (+ 13328us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.621501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.621529 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:28:00.621622 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.621629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.621812 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.622014 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:00.622922 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.624565 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.626143 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.627928 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.628069 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.628305 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.629683 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.629854 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13148,"thread_start_us":12588,"threads_started":1}
I20260811 06:28:00.630584 26436 tablet_service.cc:1467] Processing CreateTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 (DEFAULT_TABLE table=test-table [id=40fcb982b1524e15a3c214060b4ebc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:00.631928 26436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa09ba0345a74b12b820f0e2638e06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:00.633715 26764 tablet_service.cc:1467] Processing CreateTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 (DEFAULT_TABLE table=test-table [id=40fcb982b1524e15a3c214060b4ebc70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:00.635035 26764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa09ba0345a74b12b820f0e2638e06f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:00.640013 26871 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Bootstrap starting.
I20260811 06:28:00.643039 26871 tablet_bootstrap.cc:654] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:00.644264 26871 log.cc:824] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:00.648273 26873 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Bootstrap starting.
I20260811 06:28:00.650106 26871 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: No bootstrap required, opened a new log
I20260811 06:28:00.650436 26871 ts_tablet_manager.cc:1397] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260811 06:28:00.650489 26874 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Bootstrap starting.
I20260811 06:28:00.651902 26873 tablet_bootstrap.cc:654] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:00.653095 26873 log.cc:824] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:00.653155 26874 tablet_bootstrap.cc:654] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:00.654282 26874 log.cc:824] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:00.656147 26873 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: No bootstrap required, opened a new log
I20260811 06:28:00.656414 26873 ts_tablet_manager.cc:1397] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260811 06:28:00.657742 26874 tablet_bootstrap.cc:492] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: No bootstrap required, opened a new log
I20260811 06:28:00.657999 26874 ts_tablet_manager.cc:1397] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260811 06:28:00.659384 26871 raft_consensus.cc:348] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.659740 26871 raft_consensus.cc:374] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:00.659861 26871 raft_consensus.cc:729] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b3f918344634dafa113490ea1dd02f9, State: Initialized, Role: FOLLOWER
I20260811 06:28:00.660462 26871 consensus_queue.cc:260] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.662511 26680 heartbeater.cc:503] Master 127.7.4.190:44193 was elected leader, sending a full tablet report...
I20260811 06:28:00.663421 26871 ts_tablet_manager.cc:1428] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 06:28:00.667104 26873 raft_consensus.cc:348] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.667562 26873 raft_consensus.cc:374] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:00.667723 26873 raft_consensus.cc:729] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f097e59505492fa0d1f70114cc028a, State: Initialized, Role: FOLLOWER
I20260811 06:28:00.668319 26873 consensus_queue.cc:260] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.668568 26874 raft_consensus.cc:348] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.669085 26874 raft_consensus.cc:374] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:00.669196 26874 raft_consensus.cc:729] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea26b7b325ec47fa95723cb2932287ce, State: Initialized, Role: FOLLOWER
I20260811 06:28:00.669831 26874 consensus_queue.cc:260] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.670123 26873 ts_tablet_manager.cc:1428] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260811 06:28:00.670883 26845 heartbeater.cc:503] Master 127.7.4.190:44193 was elected leader, sending a full tablet report...
I20260811 06:28:00.671561 26874 ts_tablet_manager.cc:1428] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
W20260811 06:28:00.698843 26518 tablet.cc:2369] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:28:00.721967 26681 tablet.cc:2369] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:00.726912 26878 raft_consensus.cc:482] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:00.727206 26878 raft_consensus.cc:504] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.729077 26878 leader_election.cc:288] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b3f918344634dafa113490ea1dd02f9 (127.7.4.130:41569), ea26b7b325ec47fa95723cb2932287ce (127.7.4.131:34987)
I20260811 06:28:00.736485 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.729508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:43407 (local address 127.7.4.131:34987)
0811 06:28:00.729724 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.729731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.730582 (+ 851us) server_negotiation.cc:407] Connection header received
0811 06:28:00.730766 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.730778 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.730845 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.730939 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:00.731637 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.732348 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.734729 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.735539 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.735692 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.735809 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.736105 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.736225 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:28:00.736547 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.729485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:39351 (local address 127.7.4.130:41569)
0811 06:28:00.729717 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.729726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.730492 (+ 766us) server_negotiation.cc:407] Connection header received
0811 06:28:00.730708 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.730712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.730772 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.730864 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:00.731507 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.732207 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.734753 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.735600 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.735762 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.735864 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.736149 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.736274 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:28:00.736658 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.729507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:34987 (local address 127.7.4.129:43407)
0811 06:28:00.730474 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.730489 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.730634 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.730957 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.730961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.730991 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:00.731457 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.731467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.732511 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.732518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.734571 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.734586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.735755 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.735761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.735927 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.735950 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.736064 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.736185 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":69,"thread_start_us":489,"threads_started":1}
I20260811 06:28:00.736717 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.729453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:41569 (local address 127.7.4.129:39351)
0811 06:28:00.730386 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:00.730402 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:00.730565 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:00.730882 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:00.730887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:00.730911 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:00.731360 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.731369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.732333 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.732340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.734598 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:00.734610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.735839 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.735845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:00.736002 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.736024 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:28:00.736122 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:28:00.736253 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":78,"spinlock_wait_cycles":21888,"thread_start_us":593,"threads_started":1}
I20260811 06:28:00.737689 26788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa09ba0345a74b12b820f0e2638e06f5" candidate_uuid: "90f097e59505492fa0d1f70114cc028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea26b7b325ec47fa95723cb2932287ce" is_pre_election: true
I20260811 06:28:00.737699 26626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa09ba0345a74b12b820f0e2638e06f5" candidate_uuid: "90f097e59505492fa0d1f70114cc028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b3f918344634dafa113490ea1dd02f9" is_pre_election: true
I20260811 06:28:00.738149 26788 raft_consensus.cc:2454] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f097e59505492fa0d1f70114cc028a in term 0.
I20260811 06:28:00.738175 26626 raft_consensus.cc:2454] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f097e59505492fa0d1f70114cc028a in term 0.
I20260811 06:28:00.738777 26384 leader_election.cc:302] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b3f918344634dafa113490ea1dd02f9, 90f097e59505492fa0d1f70114cc028a, ea26b7b325ec47fa95723cb2932287ce; no voters:
I20260811 06:28:00.739164 26878 raft_consensus.cc:2781] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:00.739333 26878 raft_consensus.cc:482] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:00.739452 26878 raft_consensus.cc:3037] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:00.741748 26878 raft_consensus.cc:504] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.742365 26878 leader_election.cc:288] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [CANDIDATE]: Term 1 election: Requested vote from peers 2b3f918344634dafa113490ea1dd02f9 (127.7.4.130:41569), ea26b7b325ec47fa95723cb2932287ce (127.7.4.131:34987)
I20260811 06:28:00.742729 26626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa09ba0345a74b12b820f0e2638e06f5" candidate_uuid: "90f097e59505492fa0d1f70114cc028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b3f918344634dafa113490ea1dd02f9"
I20260811 06:28:00.742890 26788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa09ba0345a74b12b820f0e2638e06f5" candidate_uuid: "90f097e59505492fa0d1f70114cc028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea26b7b325ec47fa95723cb2932287ce"
I20260811 06:28:00.742949 26626 raft_consensus.cc:3037] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:00.743109 26788 raft_consensus.cc:3037] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:00.746223 26626 raft_consensus.cc:2454] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f097e59505492fa0d1f70114cc028a in term 1.
I20260811 06:28:00.746275 26788 raft_consensus.cc:2454] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f097e59505492fa0d1f70114cc028a in term 1.
I20260811 06:28:00.746671 26386 leader_election.cc:302] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b3f918344634dafa113490ea1dd02f9, 90f097e59505492fa0d1f70114cc028a; no voters:
I20260811 06:28:00.746973 26878 raft_consensus.cc:2781] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:00.747457 26878 raft_consensus.cc:686] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 1 LEADER]: Becoming Leader. State: Replica: 90f097e59505492fa0d1f70114cc028a, State: Running, Role: LEADER
I20260811 06:28:00.747917 26878 consensus_queue.cc:237] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } }
I20260811 06:28:00.752128 26295 catalog_manager.cc:5495] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f097e59505492fa0d1f70114cc028a (127.7.4.129). New cstate: current_term: 1 leader_uuid: "90f097e59505492fa0d1f70114cc028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f097e59505492fa0d1f70114cc028a" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 38497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 } health_report { overall_health: UNKNOWN } } }
I20260811 06:28:00.768991 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.763846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.7.4.129:38497)
0811 06:28:00.764007 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.764011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.764176 (+ 165us) server_negotiation.cc:407] Connection header received
0811 06:28:00.764351 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.764355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.764420 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.764535 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:00.764846 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.765506 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.767101 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.767567 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.767674 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.768093 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:00.768526 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:00.768669 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.768738 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.768797 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:28:00.776023 26788 raft_consensus.cc:1261] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 1 FOLLOWER]: Refusing update from remote peer 90f097e59505492fa0d1f70114cc028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:00.776175 26626 raft_consensus.cc:1261] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 1 FOLLOWER]: Refusing update from remote peer 90f097e59505492fa0d1f70114cc028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:00.776974 26882 consensus_queue.cc:1035] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea26b7b325ec47fa95723cb2932287ce" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:00.777280 26878 consensus_queue.cc:1035] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b3f918344634dafa113490ea1dd02f9" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:28:00.781291 26847 tablet.cc:2369] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:00.786441 26886 mvcc.cc:205] Tried to move back new op lower bound from 7317215972450123776 to 7317215972347097088. Current Snapshot: MvccSnapshot[applied={T|T < 7317215972450123776}]
I20260811 06:28:00.805054 26292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:00.805402 26292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:00.809412 26292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9eac2d86a7cf417eb228a2e4c0e8ddd8: Sending DeleteTablet for 3 replicas of tablet fa09ba0345a74b12b820f0e2638e06f5
I20260811 06:28:00.810452 26605 tablet_service.cc:1514] Processing DeleteTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:00 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60902
I20260811 06:28:00.810696 26764 tablet_service.cc:1514] Processing DeleteTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:00 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:34570
I20260811 06:28:00.810945 26436 tablet_service.cc:1514] Processing DeleteTablet for tablet fa09ba0345a74b12b820f0e2638e06f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:00 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:60530
I20260811 06:28:00.812192 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26366
I20260811 06:28:00.823055 26897 tablet_replica.cc:330] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: stopping tablet replica
I20260811 06:28:00.823745 26897 raft_consensus.cc:2229] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:00.824244 26897 raft_consensus.cc:2258] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:00.825011 26898 tablet_replica.cc:330] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: stopping tablet replica
I20260811 06:28:00.825457 26898 raft_consensus.cc:2229] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:00.826021 26898 raft_consensus.cc:2258] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:00.826606 26897 ts_tablet_manager.cc:1905] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:00.828140 26898 ts_tablet_manager.cc:1905] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:00.832902 26899 tablet_replica.cc:330] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: stopping tablet replica
I20260811 06:28:00.833403 26899 raft_consensus.cc:2229] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:00.834022 26899 raft_consensus.cc:2258] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:00.835053 26898 ts_tablet_manager.cc:1918] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:00.835259 26898 log.cc:1196] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/fa09ba0345a74b12b820f0e2638e06f5
I20260811 06:28:00.835939 26898 ts_tablet_manager.cc:1939] T fa09ba0345a74b12b820f0e2638e06f5 P ea26b7b325ec47fa95723cb2932287ce: Deleting consensus metadata
I20260811 06:28:00.835907 26897 ts_tablet_manager.cc:1918] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:00.835984 26899 ts_tablet_manager.cc:1905] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:00.836066 26897 log.cc:1196] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/fa09ba0345a74b12b820f0e2638e06f5
I20260811 06:28:00.837852 26278 catalog_manager.cc:4841] TS ea26b7b325ec47fa95723cb2932287ce (127.7.4.131:34987): tablet fa09ba0345a74b12b820f0e2638e06f5 (table test-table [id=40fcb982b1524e15a3c214060b4ebc70]) successfully deleted
I20260811 06:28:00.838802 26897 ts_tablet_manager.cc:1939] T fa09ba0345a74b12b820f0e2638e06f5 P 2b3f918344634dafa113490ea1dd02f9: Deleting consensus metadata
I20260811 06:28:00.840616 26280 catalog_manager.cc:4841] TS 2b3f918344634dafa113490ea1dd02f9 (127.7.4.130:41569): tablet fa09ba0345a74b12b820f0e2638e06f5 (table test-table [id=40fcb982b1524e15a3c214060b4ebc70]) successfully deleted
Aug 11 06:28:00 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:28:00.845224 26899 ts_tablet_manager.cc:1918] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:00.845427 26899 log.cc:1196] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/fa09ba0345a74b12b820f0e2638e06f5
I20260811 06:28:00.846036 26899 ts_tablet_manager.cc:1939] T fa09ba0345a74b12b820f0e2638e06f5 P 90f097e59505492fa0d1f70114cc028a: Deleting consensus metadata
I20260811 06:28:00.847811 26278 catalog_manager.cc:4841] TS 90f097e59505492fa0d1f70114cc028a (127.7.4.129:38497): tablet fa09ba0345a74b12b820f0e2638e06f5 (table test-table [id=40fcb982b1524e15a3c214060b4ebc70]) successfully deleted
I20260811 06:28:00.852650 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.815057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.7.4.129:38497)
0811 06:28:00.815475 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.815483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.824478 (+ 8995us) server_negotiation.cc:407] Connection header received
0811 06:28:00.826984 (+ 2506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.826989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.827083 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.827182 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:00.836251 (+ 9069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.837407 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.839562 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.840282 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.840384 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.846331 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:00.846346 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:00.846364 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:00.846410 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:00.849791 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.850595 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.850601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.850603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.850734 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.851123 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.851126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.851128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.851548 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:00.851816 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.852218 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.852405 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260811 06:28:00.853555 26511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:00.950913 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26366
I20260811 06:28:00.975260 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26525
Aug 11 06:28:00 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:28:00.991210 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:00.976032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.7.4.130:41569)
0811 06:28:00.976199 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:00.976205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:00.976280 (+ 75us) server_negotiation.cc:407] Connection header received
0811 06:28:00.976452 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:00.976457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:00.976553 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:00.976655 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:00.977921 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.979075 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.980587 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:00.981216 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:00.981334 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:00.985554 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:00.985566 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:00.985581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:00.985624 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:00.988814 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.989455 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.989461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.989463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.989583 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:00.989898 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:00.989901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:00.989903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:00.990243 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:00.990477 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:00.990694 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 06:28:00.990846 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:28:00.991950 26675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:01.080966 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26525
I20260811 06:28:01.102869 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26686
Aug 11 06:28:01 dist-test-slave-07c2 krb5kdc[26242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:28:01.118254 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:01.103717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.7.4.131:34987)
0811 06:28:01.103940 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:01.103948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:01.103974 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:28:01.104046 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:01.104050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:01.104114 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:01.104229 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:01.105540 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:01.106273 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:01.107740 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:01.108129 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:01.108260 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:01.112741 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:01.112756 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:01.112778 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:01.112811 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:01.115468 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:01.116131 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:01.116139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:01.116141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:01.116290 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:01.116673 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:01.116678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:01.116682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:01.117032 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:01.117330 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:01.117678 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:28:01.117912 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:28:01.118889 26839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:01.207517 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26686
I20260811 06:28:01.229151 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26256
I20260811 06:28:01.230320 26329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:01.345114 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26256
2026-08-11T06:28:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4443 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T06:28:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:01Z Disabled control of system clock
I20260811 06:28:01.450304 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42237
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:42237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:01.814499 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:01.814992 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:01.815191 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:01.828416 26909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:01.828599 26909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:01.828681 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:01.828785 26909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:01.828866 26909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:01.844030 26909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:42237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:01.845405 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:01.847213 26909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:01.859246 26914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:01.859200 26917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:01.859313 26915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:01.860072 26909 server_base.cc:1034] running on GCE node
I20260811 06:28:01.860812 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:01.862195 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:01.863384 26909 hybrid_clock.cc:648] HybridClock initialized: now 1786429681863331 us; error 38 us; skew 500 ppm
I20260811 06:28:01.863760 26909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:01.865981 26909 webserver.cc:460] Webserver started at http://127.7.4.190:44685/ using document root <none> and password file <none>
I20260811 06:28:01.866714 26909 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:01.866832 26909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:01.867178 26909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:01.870260 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "9995283469374173845bf1b3fd3e2329"
format_stamp: "Formatted at 2026-08-11 06:28:01 on dist-test-slave-07c2"
server_key: "ac5e4b7e1fec6456900fc1026d44c2ce"
server_key_iv: "d87b5be25548dc606161411b1d7e5c36"
server_key_version: "encryptionkey@0"
I20260811 06:28:01.871081 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "9995283469374173845bf1b3fd3e2329"
format_stamp: "Formatted at 2026-08-11 06:28:01 on dist-test-slave-07c2"
server_key: "ac5e4b7e1fec6456900fc1026d44c2ce"
server_key_iv: "d87b5be25548dc606161411b1d7e5c36"
server_key_version: "encryptionkey@0"
I20260811 06:28:01.876243 26909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:28:01.879734 26925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:01.881004 26909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:28:01.881193 26909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "9995283469374173845bf1b3fd3e2329"
format_stamp: "Formatted at 2026-08-11 06:28:01 on dist-test-slave-07c2"
server_key: "ac5e4b7e1fec6456900fc1026d44c2ce"
server_key_iv: "d87b5be25548dc606161411b1d7e5c36"
server_key_version: "encryptionkey@0"
I20260811 06:28:01.881421 26909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:01.896049 26909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:01.896898 26909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:01.897233 26909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:01.915729 26909 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:42237
I20260811 06:28:01.915766 26986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:42237 every 8 connection(s)
I20260811 06:28:01.917126 26909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:01.920809 26988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:01.922979 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 26909
I20260811 06:28:01.923161 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:01.923651 7186 external_mini_cluster.cc:1428] Setting key 8674615435c64e7cba25eb28476ee8e4
I20260811 06:28:01.932364 26988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: Bootstrap starting.
I20260811 06:28:01.936053 26988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:01.937141 26988 log.cc:824] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:01.937713 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:01.925525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.7.4.190:42237)
0811 06:28:01.926565 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:01.926576 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:01.926620 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:28:01.928368 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:01.928390 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:01.928854 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:01.929170 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:01.929829 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:01.931123 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:01.933121 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:01.933752 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:01.933901 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:01.935157 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:01.935164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:01.935175 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:01.935749 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:01.935843 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:01.935864 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:01.935962 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:01.936225 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 06:28:01.936562 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":421,"threads_started":1}
I20260811 06:28:01.939973 26988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: No bootstrap required, opened a new log
I20260811 06:28:01.947206 26988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } }
I20260811 06:28:01.947602 26988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:01.947713 26988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9995283469374173845bf1b3fd3e2329, State: Initialized, Role: FOLLOWER
I20260811 06:28:01.948225 26988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } }
I20260811 06:28:01.948473 26988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:01.948618 26988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:01.948805 26988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:01.950856 26988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } }
I20260811 06:28:01.951404 26988 leader_election.cc:302] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9995283469374173845bf1b3fd3e2329; no voters:
I20260811 06:28:01.951960 26988 leader_election.cc:288] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:01.952109 26994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:01.952904 26994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [term 1 LEADER]: Becoming Leader. State: Replica: 9995283469374173845bf1b3fd3e2329, State: Running, Role: LEADER
I20260811 06:28:01.953423 26994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } }
I20260811 06:28:01.953958 26988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:01.956053 26995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9995283469374173845bf1b3fd3e2329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } } }
I20260811 06:28:01.956308 26997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9995283469374173845bf1b3fd3e2329. Latest consensus state: current_term: 1 leader_uuid: "9995283469374173845bf1b3fd3e2329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9995283469374173845bf1b3fd3e2329" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 42237 } } }
I20260811 06:28:01.956529 26995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:01.956583 26997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:01.959160 27005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:01.964183 27005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:01.972306 27005 catalog_manager.cc:1347] Generated new cluster ID: e3de2d6ba17e4ab0bf25a50d6923d766
I20260811 06:28:01.972420 27005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:02.006310 27005 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:02.007725 27005 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:02.017506 27005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: Generated new TSK 0
I20260811 06:28:02.018289 27005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:02.023927 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--builtin_ntp_servers=127.7.4.148:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:02.368973 27016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:02.369235 27016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:02.369390 27016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:02.376513 27016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:02.376618 27016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:02.376798 27016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:28:02.385864 27016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:02.387121 27016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:02.388967 27016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:02.402220 27021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:02.402488 27022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:02.402943 27024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:02.403754 27016 server_base.cc:1034] running on GCE node
I20260811 06:28:02.404405 27016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:02.405289 27016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:02.406491 27016 hybrid_clock.cc:648] HybridClock initialized: now 1786429682406441 us; error 37 us; skew 500 ppm
I20260811 06:28:02.406759 27016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:02.409091 27016 webserver.cc:460] Webserver started at http://127.7.4.129:42053/ using document root <none> and password file <none>
I20260811 06:28:02.409750 27016 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:02.409843 27016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:02.410120 27016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:02.413002 27016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "84bb982ea5114b429460a59946088ccc"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "fdc679892df94e3cc8cbe27200e01fb8"
server_key_iv: "799abb156d53709bd84a91b42ec7a5ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.413749 27016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "84bb982ea5114b429460a59946088ccc"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "fdc679892df94e3cc8cbe27200e01fb8"
server_key_iv: "799abb156d53709bd84a91b42ec7a5ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.418653 27016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 06:28:02.422120 27031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.423489 27016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:02.423653 27016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "84bb982ea5114b429460a59946088ccc"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "fdc679892df94e3cc8cbe27200e01fb8"
server_key_iv: "799abb156d53709bd84a91b42ec7a5ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.423866 27016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:02.440764 27016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:02.441560 27016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:02.441835 27016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:02.442811 27016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:02.444295 27016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:02.444423 27016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.444541 27016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:02.444628 27016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.478718 27016 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:39405
I20260811 06:28:02.478806 27164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:39405 every 8 connection(s)
I20260811 06:28:02.480064 27016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:28:02.488703 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27016
I20260811 06:28:02.488899 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:28:02.489339 7186 external_mini_cluster.cc:1428] Setting key d7ec53a307d36416e2e1c8582aca3592
I20260811 06:28:02.492069 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--builtin_ntp_servers=127.7.4.148:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:02.492527 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:02.482576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:34363 (local address 127.7.4.190:42237)
0811 06:28:02.483055 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:02.483063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:02.484246 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 06:28:02.485985 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:02.485989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:02.486196 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:02.486311 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:02.487266 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.488019 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.489736 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.490139 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.490283 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:02.491676 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:02.491680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:02.491683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:02.491700 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:02.491743 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:02.491755 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:02.491848 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:02.492063 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 06:28:02.492173 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260811 06:28:02.493494 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:02.483157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42237 (local address 127.7.4.129:34363)
0811 06:28:02.484102 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:02.484137 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:02.485749 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:02.486491 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:02.486498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:02.486582 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:02.487123 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:02.487139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.488160 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.488164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:02.489580 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:02.489593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.490381 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.490390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:02.490624 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:02.491363 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:02.491366 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:02.491405 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:02.491480 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:02.491852 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:02.491865 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:02.491875 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:28:02.492048 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 06:28:02.492430 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":340,"threads_started":1}
I20260811 06:28:02.495015 27165 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42237
I20260811 06:28:02.495360 27165 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:02.496430 27165 heartbeater.cc:511] Master 127.7.4.190:42237 requested a full tablet report, sending...
I20260811 06:28:02.498446 26944 ts_manager.cc:194] Registered new tserver with Master: 84bb982ea5114b429460a59946088ccc (127.7.4.129:39405)
I20260811 06:28:02.500012 26944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.129:34363
W20260811 06:28:02.840163 27169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:02.840420 27169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:02.840592 27169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:02.847664 27169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:02.847756 27169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:02.847999 27169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:28:02.857071 27169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:02.858317 27169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:02.860124 27169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:02.873062 27177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:02.873092 27180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:02.873133 27176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:02.874282 27169 server_base.cc:1034] running on GCE node
I20260811 06:28:02.874953 27169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:02.875895 27169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:02.877120 27169 hybrid_clock.cc:648] HybridClock initialized: now 1786429682877082 us; error 47 us; skew 500 ppm
I20260811 06:28:02.877426 27169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:02.879663 27169 webserver.cc:460] Webserver started at http://127.7.4.130:33825/ using document root <none> and password file <none>
I20260811 06:28:02.880316 27169 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:02.880453 27169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:02.880772 27169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:02.883618 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "cfbe9d7ba7a04774828fa3cfe93440dd"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "ec0f45814f1773a2a545266fe83e0fe2"
server_key_iv: "33f159649bb86e2edfae19a8c05e4797"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.884373 27169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "cfbe9d7ba7a04774828fa3cfe93440dd"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "ec0f45814f1773a2a545266fe83e0fe2"
server_key_iv: "33f159649bb86e2edfae19a8c05e4797"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.889403 27169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:28:02.892704 27188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.893905 27169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:28:02.894075 27169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "cfbe9d7ba7a04774828fa3cfe93440dd"
format_stamp: "Formatted at 2026-08-11 06:28:02 on dist-test-slave-07c2"
server_key: "ec0f45814f1773a2a545266fe83e0fe2"
server_key_iv: "33f159649bb86e2edfae19a8c05e4797"
server_key_version: "encryptionkey@0"
I20260811 06:28:02.894311 27169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:02.909933 27169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:02.910660 27169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:02.910919 27169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:02.911823 27169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:02.913105 27169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:02.913224 27169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.913331 27169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:02.913409 27169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:02.951177 27169 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:45921
I20260811 06:28:02.951232 27316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:45921 every 8 connection(s)
I20260811 06:28:02.952585 27169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:28:02.955893 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27169
I20260811 06:28:02.956070 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:28:02.956733 7186 external_mini_cluster.cc:1428] Setting key c6256fab653d59888f6f0c45c21425c8
I20260811 06:28:02.959909 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--builtin_ntp_servers=127.7.4.148:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:02.966403 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:02.955335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:49419 (local address 127.7.4.190:42237)
0811 06:28:02.955501 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:02.955505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:02.957241 (+ 1736us) server_negotiation.cc:407] Connection header received
0811 06:28:02.959382 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:02.959388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:02.959476 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:02.959606 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:02.960678 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.961946 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.963486 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.964245 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.964398 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:02.965671 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:02.965676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:02.965679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:02.965696 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:02.965727 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:02.965757 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:02.965829 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:02.966105 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 06:28:02.966184 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:28:02.967957 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:02.955914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42237 (local address 127.7.4.130:49419)
0811 06:28:02.957051 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:02.957098 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:02.959085 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:02.959776 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:02.959784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:02.959860 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:02.960477 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:02.960494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.962097 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.962101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:02.963283 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:02.963342 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:02.964436 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:02.964441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:02.964620 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:02.965330 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:02.965337 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:02.965390 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:02.965487 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:02.965859 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:02.965874 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:02.965885 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:28:02.966104 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 06:28:02.966518 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":354,"threads_started":1}
I20260811 06:28:02.969312 27317 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42237
I20260811 06:28:02.969632 27317 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:02.970605 27317 heartbeater.cc:511] Master 127.7.4.190:42237 requested a full tablet report, sending...
I20260811 06:28:02.972088 26944 ts_manager.cc:194] Registered new tserver with Master: cfbe9d7ba7a04774828fa3cfe93440dd (127.7.4.130:45921)
I20260811 06:28:02.972864 26944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.130:49419
W20260811 06:28:03.295358 27321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:03.295617 27321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:03.295804 27321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:03.302776 27321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:03.302884 27321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:03.303048 27321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:03.311754 27321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:42237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:03.313339 27321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:03.315517 27321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:03.328058 27327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:03.327981 27329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:03.328022 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:03.329180 27321 server_base.cc:1034] running on GCE node
I20260811 06:28:03.329854 27321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:03.330863 27321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:03.332089 27321 hybrid_clock.cc:648] HybridClock initialized: now 1786429683332044 us; error 53 us; skew 500 ppm
I20260811 06:28:03.332405 27321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:03.334599 27321 webserver.cc:460] Webserver started at http://127.7.4.131:39669/ using document root <none> and password file <none>
I20260811 06:28:03.335271 27321 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:03.335464 27321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:03.335783 27321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:03.338480 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "3808eaa983ee4123a215c1925af312cb"
format_stamp: "Formatted at 2026-08-11 06:28:03 on dist-test-slave-07c2"
server_key: "bdcf1e5ab2a6bd7b6a2e50323870e73f"
server_key_iv: "2e66347b37e2062b5d82f3c212a6d46f"
server_key_version: "encryptionkey@0"
I20260811 06:28:03.339241 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "3808eaa983ee4123a215c1925af312cb"
format_stamp: "Formatted at 2026-08-11 06:28:03 on dist-test-slave-07c2"
server_key: "bdcf1e5ab2a6bd7b6a2e50323870e73f"
server_key_iv: "2e66347b37e2062b5d82f3c212a6d46f"
server_key_version: "encryptionkey@0"
I20260811 06:28:03.344200 27321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:28:03.347474 27338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:03.348712 27321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:28:03.348877 27321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "3808eaa983ee4123a215c1925af312cb"
format_stamp: "Formatted at 2026-08-11 06:28:03 on dist-test-slave-07c2"
server_key: "bdcf1e5ab2a6bd7b6a2e50323870e73f"
server_key_iv: "2e66347b37e2062b5d82f3c212a6d46f"
server_key_version: "encryptionkey@0"
I20260811 06:28:03.349086 27321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:03.375764 27321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:03.376554 27321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:03.376824 27321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:03.377755 27321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:03.379132 27321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:03.379253 27321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:03.379441 27321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:03.379534 27321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:03.413084 27321 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:40923
I20260811 06:28:03.413165 27463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:40923 every 8 connection(s)
I20260811 06:28:03.414345 27321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:03.423190 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27321
I20260811 06:28:03.423399 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:03.423931 7186 external_mini_cluster.cc:1428] Setting key 97e53470988c975140047a18125acd15
I20260811 06:28:03.427348 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.416787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:33121 (local address 127.7.4.190:42237)
0811 06:28:03.417007 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.417014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.418377 (+ 1363us) server_negotiation.cc:407] Connection header received
0811 06:28:03.420243 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.420247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.420314 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.420400 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:03.421377 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.422114 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.424211 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.424739 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.424833 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.426394 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:03.426400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:03.426405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:03.426424 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:03.426465 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:03.426482 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:03.426577 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.426847 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.426945 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:28:03.428254 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.417320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:42237 (local address 127.7.4.131:33121)
0811 06:28:03.418224 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.418265 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.420010 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.420615 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.420622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.420701 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:03.421226 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.421241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.422259 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.422263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.424068 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.424086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.424950 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.424956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.425285 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.426070 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:03.426074 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:03.426120 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:03.426189 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:03.426594 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:03.426602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:03.426608 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.426804 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.427185 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":308,"threads_started":1}
I20260811 06:28:03.429457 27464 heartbeater.cc:347] Connected to a master server at 127.7.4.190:42237
I20260811 06:28:03.429711 27464 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:03.430364 27464 heartbeater.cc:511] Master 127.7.4.190:42237 requested a full tablet report, sending...
I20260811 06:28:03.431627 26944 ts_manager.cc:194] Registered new tserver with Master: 3808eaa983ee4123a215c1925af312cb (127.7.4.131:40923)
I20260811 06:28:03.432379 26944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.131:33121
I20260811 06:28:03.439047 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:03.448828 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.442939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.7.4.190:42237)
0811 06:28:03.443126 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.443130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.443642 (+ 512us) server_negotiation.cc:407] Connection header received
0811 06:28:03.443846 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.443851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.443951 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.444075 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:03.444573 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.445260 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.446615 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.447030 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.447160 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.448114 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:03.448120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:03.448128 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:03.448145 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:03.448174 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:03.448186 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:03.448285 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.448411 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.448498 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:28:03.452196 26944 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:03.459656 26944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:03.482125 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.471023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:39405 (local address 127.0.0.1:43816)
0811 06:28:03.472140 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.472189 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.472433 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.474111 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.474121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.474163 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:03.474623 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.474639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.476698 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.476706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.478999 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.479014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.481245 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.481252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.481443 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.481492 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.481615 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.481805 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":571,"threads_started":1}
I20260811 06:28:03.483083 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.471224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.7.4.129:39405)
0811 06:28:03.473404 (+ 2180us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.473415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.473453 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:28:03.473570 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.473580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.473860 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.474068 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:03.474875 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.476517 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.479217 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.480673 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.480818 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.481096 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.482533 (+ 1437us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.482706 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2010,"thread_start_us":1857,"threads_started":1}
I20260811 06:28:03.487471 27088 tablet_service.cc:1467] Processing CreateTablet for tablet 9996fdd689b3465f800c8781fcf725e9 (DEFAULT_TABLE table=test-table [id=839947347d1c446d9e72041cbf085b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:03.489104 27088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9996fdd689b3465f800c8781fcf725e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:03.493010 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.475169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:45921 (local address 127.0.0.1:58470)
0811 06:28:03.484703 (+ 9534us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.484721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.484841 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.485787 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.485792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.485816 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:03.485995 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.486007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.487820 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.487825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.489907 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.489923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.491977 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.491985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.492281 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.492387 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.492488 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.492635 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9409,"mutex_wait_us":130,"thread_start_us":11699,"threads_started":1}
I20260811 06:28:03.493135 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.471709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.7.4.131:40923)
0811 06:28:03.474748 (+ 3039us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.474759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.475328 (+ 569us) server_negotiation.cc:407] Connection header received
0811 06:28:03.475434 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.475441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.475659 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.475825 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:03.476763 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.487735 (+ 10972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.490183 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.491569 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.491702 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.491962 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.492588 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.492754 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2865,"thread_start_us":2708,"threads_started":1}
I20260811 06:28:03.493083 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.471439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40923 (local address 127.0.0.1:53786)
0811 06:28:03.475012 (+ 3573us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.475031 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.475159 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.476115 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.476119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.476143 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:03.476519 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.476530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.487923 (+ 11393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.487928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.489922 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.489935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.492176 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.492186 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.492348 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.492414 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.492539 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.492689 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3447,"mutex_wait_us":43,"thread_start_us":3318,"threads_started":1}
I20260811 06:28:03.493727 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.475456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58470 (local address 127.7.4.130:45921)
0811 06:28:03.483640 (+ 8184us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.483646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.484970 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 06:28:03.485086 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.485097 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.485337 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.485527 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:03.486185 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.487645 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.490085 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.491569 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.491702 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.491928 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.493256 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.493424 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8036,"thread_start_us":7927,"threads_started":1}
I20260811 06:28:03.497370 27242 tablet_service.cc:1467] Processing CreateTablet for tablet 9996fdd689b3465f800c8781fcf725e9 (DEFAULT_TABLE table=test-table [id=839947347d1c446d9e72041cbf085b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:03.497370 27391 tablet_service.cc:1467] Processing CreateTablet for tablet 9996fdd689b3465f800c8781fcf725e9 (DEFAULT_TABLE table=test-table [id=839947347d1c446d9e72041cbf085b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:03.498823 27242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9996fdd689b3465f800c8781fcf725e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:03.498818 27391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9996fdd689b3465f800c8781fcf725e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:03.502725 27165 heartbeater.cc:503] Master 127.7.4.190:42237 was elected leader, sending a full tablet report...
I20260811 06:28:03.507372 27489 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Bootstrap starting.
I20260811 06:28:03.510926 27489 tablet_bootstrap.cc:654] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:03.512526 27489 log.cc:824] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:03.514595 27493 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Bootstrap starting.
I20260811 06:28:03.515216 27494 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Bootstrap starting.
I20260811 06:28:03.515838 27489 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: No bootstrap required, opened a new log
I20260811 06:28:03.516117 27489 ts_tablet_manager.cc:1397] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:28:03.518332 27494 tablet_bootstrap.cc:654] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:03.518343 27493 tablet_bootstrap.cc:654] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:03.519326 27494 log.cc:824] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:03.519681 27493 log.cc:824] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:03.522728 27494 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: No bootstrap required, opened a new log
I20260811 06:28:03.522801 27493 tablet_bootstrap.cc:492] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: No bootstrap required, opened a new log
I20260811 06:28:03.523012 27494 ts_tablet_manager.cc:1397] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 06:28:03.523043 27493 ts_tablet_manager.cc:1397] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:28:03.527011 27489 raft_consensus.cc:348] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.527557 27489 raft_consensus.cc:374] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:03.527716 27489 raft_consensus.cc:729] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bb982ea5114b429460a59946088ccc, State: Initialized, Role: FOLLOWER
I20260811 06:28:03.528427 27489 consensus_queue.cc:260] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.531244 27489 ts_tablet_manager.cc:1428] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 06:28:03.531819 27493 raft_consensus.cc:348] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.532286 27493 raft_consensus.cc:374] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:03.532441 27493 raft_consensus.cc:729] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbe9d7ba7a04774828fa3cfe93440dd, State: Initialized, Role: FOLLOWER
I20260811 06:28:03.533164 27493 consensus_queue.cc:260] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.533679 27494 raft_consensus.cc:348] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.534160 27494 raft_consensus.cc:374] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:03.534312 27317 heartbeater.cc:503] Master 127.7.4.190:42237 was elected leader, sending a full tablet report...
I20260811 06:28:03.534310 27494 raft_consensus.cc:729] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3808eaa983ee4123a215c1925af312cb, State: Initialized, Role: FOLLOWER
I20260811 06:28:03.534963 27493 ts_tablet_manager.cc:1428] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260811 06:28:03.535003 27494 consensus_queue.cc:260] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.536362 27464 heartbeater.cc:503] Master 127.7.4.190:42237 was elected leader, sending a full tablet report...
I20260811 06:28:03.537096 27494 ts_tablet_manager.cc:1428] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260811 06:28:03.666010 27465 tablet.cc:2369] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:28:03.704813 27319 tablet.cc:2369] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:03.712014 27497 raft_consensus.cc:482] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:03.712288 27497 raft_consensus.cc:504] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.713932 27497 leader_election.cc:288] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3808eaa983ee4123a215c1925af312cb (127.7.4.131:40923), cfbe9d7ba7a04774828fa3cfe93440dd (127.7.4.130:45921)
I20260811 06:28:03.721217 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.714254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:38217 (local address 127.7.4.131:40923)
0811 06:28:03.714436 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.714440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.714926 (+ 486us) server_negotiation.cc:407] Connection header received
0811 06:28:03.715173 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.715178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.715264 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.715418 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:03.716166 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.717199 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.719338 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.720244 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.720343 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.720414 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.720815 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.720961 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:28:03.721289 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.714166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:40923 (local address 127.7.4.129:38217)
0811 06:28:03.714823 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.714843 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.714985 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.715593 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.715597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.715624 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:03.716005 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.716019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.717328 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.717334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.719126 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.719139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.720433 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.720439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.720617 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.720649 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.720757 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.720906 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":79,"thread_start_us":337,"threads_started":1}
I20260811 06:28:03.721376 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.715279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:46215 (local address 127.7.4.130:45921)
0811 06:28:03.715491 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.715496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.715889 (+ 393us) server_negotiation.cc:407] Connection header received
0811 06:28:03.716033 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.716038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.716138 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.716217 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:03.716861 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.717738 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.719650 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.720423 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.720554 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.720640 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.720963 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.721101 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:28:03.721400 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.714976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:45921 (local address 127.7.4.129:46215)
0811 06:28:03.715784 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:03.715802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:03.715904 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:03.716274 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:03.716278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:03.716306 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:03.716708 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.716719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.717888 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.717891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.719455 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:03.719469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.720606 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.720610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:03.720790 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.720821 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:28:03.720913 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:28:03.721026 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":62,"thread_start_us":518,"threads_started":1}
I20260811 06:28:03.722326 27265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9996fdd689b3465f800c8781fcf725e9" candidate_uuid: "84bb982ea5114b429460a59946088ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" is_pre_election: true
I20260811 06:28:03.722481 27414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9996fdd689b3465f800c8781fcf725e9" candidate_uuid: "84bb982ea5114b429460a59946088ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3808eaa983ee4123a215c1925af312cb" is_pre_election: true
I20260811 06:28:03.722872 27265 raft_consensus.cc:2454] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bb982ea5114b429460a59946088ccc in term 0.
I20260811 06:28:03.722997 27414 raft_consensus.cc:2454] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84bb982ea5114b429460a59946088ccc in term 0.
I20260811 06:28:03.723604 27034 leader_election.cc:302] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84bb982ea5114b429460a59946088ccc, cfbe9d7ba7a04774828fa3cfe93440dd; no voters:
I20260811 06:28:03.724018 27497 raft_consensus.cc:2781] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:03.724169 27497 raft_consensus.cc:482] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:03.724308 27497 raft_consensus.cc:3037] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:03.726512 27497 raft_consensus.cc:504] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
I20260811 06:28:03.727100 27497 leader_election.cc:288] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [CANDIDATE]: Term 1 election: Requested vote from peers 3808eaa983ee4123a215c1925af312cb (127.7.4.131:40923), cfbe9d7ba7a04774828fa3cfe93440dd (127.7.4.130:45921)
I20260811 06:28:03.727536 27414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9996fdd689b3465f800c8781fcf725e9" candidate_uuid: "84bb982ea5114b429460a59946088ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3808eaa983ee4123a215c1925af312cb"
I20260811 06:28:03.727643 27265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9996fdd689b3465f800c8781fcf725e9" candidate_uuid: "84bb982ea5114b429460a59946088ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd"
I20260811 06:28:03.727727 27414 raft_consensus.cc:3037] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:03.727871 27265 raft_consensus.cc:3037] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:03.729924 27414 raft_consensus.cc:2454] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bb982ea5114b429460a59946088ccc in term 1.
I20260811 06:28:03.730229 27265 raft_consensus.cc:2454] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84bb982ea5114b429460a59946088ccc in term 1.
I20260811 06:28:03.730288 27034 leader_election.cc:302] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3808eaa983ee4123a215c1925af312cb, 84bb982ea5114b429460a59946088ccc; no voters:
I20260811 06:28:03.730593 27497 raft_consensus.cc:2781] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:03.731030 27497 raft_consensus.cc:686] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 1 LEADER]: Becoming Leader. State: Replica: 84bb982ea5114b429460a59946088ccc, State: Running, Role: LEADER
I20260811 06:28:03.731572 27497 consensus_queue.cc:237] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } }
W20260811 06:28:03.732478 27166 tablet.cc:2369] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:03.736495 26944 catalog_manager.cc:5495] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc reported cstate change: term changed from 0 to 1, leader changed from <none> to 84bb982ea5114b429460a59946088ccc (127.7.4.129). New cstate: current_term: 1 leader_uuid: "84bb982ea5114b429460a59946088ccc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84bb982ea5114b429460a59946088ccc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 39405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 } health_report { overall_health: UNKNOWN } } }
I20260811 06:28:03.776556 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.770973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.7.4.129:39405)
0811 06:28:03.771172 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.771178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.771198 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:28:03.771351 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.771362 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.771464 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.771611 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:03.771996 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.772662 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.774211 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.774638 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.774801 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.775390 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:03.775897 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:03.776067 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.776198 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.776280 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:28:03.784260 27265 raft_consensus.cc:1261] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 1 FOLLOWER]: Refusing update from remote peer 84bb982ea5114b429460a59946088ccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:03.784301 27414 raft_consensus.cc:1261] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 1 FOLLOWER]: Refusing update from remote peer 84bb982ea5114b429460a59946088ccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:03.785236 27502 consensus_queue.cc:1035] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3808eaa983ee4123a215c1925af312cb" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:03.785605 27497 consensus_queue.cc:1035] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfbe9d7ba7a04774828fa3cfe93440dd" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:03.808861 26943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:03.809130 26943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:03.812906 26943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9995283469374173845bf1b3fd3e2329: Sending DeleteTablet for 3 replicas of tablet 9996fdd689b3465f800c8781fcf725e9
I20260811 06:28:03.813829 27088 tablet_service.cc:1514] Processing DeleteTablet for tablet 9996fdd689b3465f800c8781fcf725e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:03 UTC) from {username='slave'} at 127.0.0.1:43816
I20260811 06:28:03.814180 27242 tablet_service.cc:1514] Processing DeleteTablet for tablet 9996fdd689b3465f800c8781fcf725e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:03 UTC) from {username='slave'} at 127.0.0.1:58470
I20260811 06:28:03.814384 27391 tablet_service.cc:1514] Processing DeleteTablet for tablet 9996fdd689b3465f800c8781fcf725e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:03 UTC) from {username='slave'} at 127.0.0.1:53786
I20260811 06:28:03.814738 27516 tablet_replica.cc:330] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: stopping tablet replica
I20260811 06:28:03.815263 27516 raft_consensus.cc:2229] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:03.815500 27517 tablet_replica.cc:330] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: stopping tablet replica
I20260811 06:28:03.815925 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27016
I20260811 06:28:03.816056 27518 tablet_replica.cc:330] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: stopping tablet replica
I20260811 06:28:03.816128 27516 raft_consensus.cc:2258] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:03.816416 27518 raft_consensus.cc:2229] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:03.816864 27518 raft_consensus.cc:2258] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:03.817482 27517 raft_consensus.cc:2229] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:03.817951 27517 raft_consensus.cc:2258] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:03.820204 27517 ts_tablet_manager.cc:1905] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:03.820441 27518 ts_tablet_manager.cc:1905] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:03.825479 27516 ts_tablet_manager.cc:1905] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:03.827432 27517 ts_tablet_manager.cc:1918] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:03.827653 27517 log.cc:1196] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/9996fdd689b3465f800c8781fcf725e9
I20260811 06:28:03.828446 27517 ts_tablet_manager.cc:1939] T 9996fdd689b3465f800c8781fcf725e9 P cfbe9d7ba7a04774828fa3cfe93440dd: Deleting consensus metadata
I20260811 06:28:03.829155 27518 ts_tablet_manager.cc:1918] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:03.829380 27518 log.cc:1196] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/9996fdd689b3465f800c8781fcf725e9
I20260811 06:28:03.829941 27518 ts_tablet_manager.cc:1939] T 9996fdd689b3465f800c8781fcf725e9 P 3808eaa983ee4123a215c1925af312cb: Deleting consensus metadata
I20260811 06:28:03.830303 26928 catalog_manager.cc:4841] TS cfbe9d7ba7a04774828fa3cfe93440dd (127.7.4.130:45921): tablet 9996fdd689b3465f800c8781fcf725e9 (table test-table [id=839947347d1c446d9e72041cbf085b54]) successfully deleted
I20260811 06:28:03.831725 26928 catalog_manager.cc:4841] TS 3808eaa983ee4123a215c1925af312cb (127.7.4.131:40923): tablet 9996fdd689b3465f800c8781fcf725e9 (table test-table [id=839947347d1c446d9e72041cbf085b54]) successfully deleted
I20260811 06:28:03.835104 27516 ts_tablet_manager.cc:1918] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:03.835271 27516 log.cc:1196] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/9996fdd689b3465f800c8781fcf725e9
I20260811 06:28:03.835867 27516 ts_tablet_manager.cc:1939] T 9996fdd689b3465f800c8781fcf725e9 P 84bb982ea5114b429460a59946088ccc: Deleting consensus metadata
I20260811 06:28:03.837426 26927 catalog_manager.cc:4841] TS 84bb982ea5114b429460a59946088ccc (127.7.4.129:39405): tablet 9996fdd689b3465f800c8781fcf725e9 (table test-table [id=839947347d1c446d9e72041cbf085b54]) successfully deleted
I20260811 06:28:03.840482 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.817857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.7.4.129:39405)
0811 06:28:03.818034 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.818041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.832649 (+ 14608us) server_negotiation.cc:407] Connection header received
0811 06:28:03.832823 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.832829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.832936 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.833057 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:03.833563 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.834841 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.836877 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.837578 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.837714 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.838805 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:03.838815 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:03.838833 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:03.839619 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:03.839676 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:03.839701 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:03.839822 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.839999 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.840105 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:28:03.841224 27159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:03.947261 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27016
I20260811 06:28:03.977108 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27169
I20260811 06:28:03.984742 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:03.977998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.7.4.130:45921)
0811 06:28:03.978192 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:03.978199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:03.978221 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:28:03.978282 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:03.978288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:03.978375 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:03.978475 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:03.978841 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.980064 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.981339 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:03.981990 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:03.982105 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:03.983146 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:03.983156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:03.983172 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:03.983909 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:03.983965 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:03.983989 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:03.984114 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:03.984238 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 06:28:03.984341 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:03.985589 27312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:04.075279 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27169
I20260811 06:28:04.096012 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27321
I20260811 06:28:04.102995 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:04.096796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.7.4.131:40923)
0811 06:28:04.096964 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:04.096969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:04.096988 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:28:04.097179 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:04.097187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:04.097319 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:04.097451 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:04.097973 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:04.098712 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:04.100154 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:04.100557 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:04.100702 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:04.101673 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:04.101683 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:04.101697 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:04.102257 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:04.102308 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:04.102329 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:04.102453 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:04.102582 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:28:04.102687 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:28:04.103698 27457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:04.191079 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27321
I20260811 06:28:04.211261 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 26909
I20260811 06:28:04.212422 26980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:04.320624 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 26909
2026-08-11T06:28:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2969 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:04 dist-test-slave-07c2 krb5kdc[27527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:04 dist-test-slave-07c2 krb5kdc[27527](info): set up 2 sockets
Aug 11 06:28:04 dist-test-slave-07c2 krb5kdc[27527](info): commencing operation
krb5kdc: starting...
W20260811 06:28:05.480827 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:05 dist-test-slave-07c2 krb5kdc[27527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:05.774567 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46071
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:06.102424 27544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:06.102751 27544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:06.102924 27544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:06.109881 27544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:06.109999 27544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:06.110093 27544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:06.110185 27544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:06.110280 27544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:06.119446 27544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:06.120656 27544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:06.121950 27544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:06.133257 27549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:06.133333 27554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:06.133358 27550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:06.134305 27544 server_base.cc:1034] running on GCE node
I20260811 06:28:06.135082 27544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:06.136444 27544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:06.137673 27544 hybrid_clock.cc:648] HybridClock initialized: now 1786429686137583 us; error 71 us; skew 500 ppm
I20260811 06:28:06.138010 27544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:06 dist-test-slave-07c2 krb5kdc[27527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429686, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:06.141161 27544 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:06.142509 27544 webserver.cc:460] Webserver started at http://127.7.4.190:40405/ using document root <none> and password file <none>
I20260811 06:28:06.143107 27544 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:06.143204 27544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:06.143535 27544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:06.146353 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "dae1a42f9a12430da53f18d2ae4074ad"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "163326ad3606863d1e0a06fa9cd0ec44"
server_key_iv: "6ac126fbd17763d362f19aa54143b42e"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.147058 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "dae1a42f9a12430da53f18d2ae4074ad"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "163326ad3606863d1e0a06fa9cd0ec44"
server_key_iv: "6ac126fbd17763d362f19aa54143b42e"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.151912 27544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:28:06.155055 27563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:06.156342 27544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:06.156531 27544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "dae1a42f9a12430da53f18d2ae4074ad"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "163326ad3606863d1e0a06fa9cd0ec44"
server_key_iv: "6ac126fbd17763d362f19aa54143b42e"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.156723 27544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:06.167810 27544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:06.171914 27544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:06.172202 27544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:06.190400 27544 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:46071
I20260811 06:28:06.190454 27622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:46071 every 8 connection(s)
I20260811 06:28:06.191694 27544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:06.195137 27625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:06.200171 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27544
I20260811 06:28:06.200359 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:06.200932 7186 external_mini_cluster.cc:1428] Setting key 3c190c871c2cac1734202cd0b6fac66e
I20260811 06:28:06.205816 27625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: Bootstrap starting.
I20260811 06:28:06.209439 27625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:06.210597 27625 log.cc:824] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:06.213882 27625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: No bootstrap required, opened a new log
Aug 11 06:28:06 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:06.224290 27625 raft_consensus.cc:348] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } }
I20260811 06:28:06.224710 27625 raft_consensus.cc:374] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:06.224862 27625 raft_consensus.cc:729] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae1a42f9a12430da53f18d2ae4074ad, State: Initialized, Role: FOLLOWER
I20260811 06:28:06.225525 27625 consensus_queue.cc:260] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } }
I20260811 06:28:06.225844 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:06.202798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.7.4.190:46071)
0811 06:28:06.203610 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:06.203620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:06.203665 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:28:06.205296 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:06.205316 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:06.205636 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:06.205970 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:06.207792 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.209092 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.210986 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.212065 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.212206 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:06.217095 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:06.217105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:06.217124 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:06.217169 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:06.221205 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:06.222027 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:06.222036 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:06.222044 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:06.222255 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:06.222664 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:06.222668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:06.222672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:06.223379 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:06.223647 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:06.224031 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 06:28:06.224450 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":397,"threads_started":1}
I20260811 06:28:06.225841 27625 raft_consensus.cc:388] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:06.226089 27625 raft_consensus.cc:482] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:06.226315 27625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:06.229192 27625 raft_consensus.cc:504] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } }
I20260811 06:28:06.229651 27625 leader_election.cc:302] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dae1a42f9a12430da53f18d2ae4074ad; no voters:
I20260811 06:28:06.230237 27625 leader_election.cc:288] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:06.230337 27631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:06.231130 27631 raft_consensus.cc:686] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [term 1 LEADER]: Becoming Leader. State: Replica: dae1a42f9a12430da53f18d2ae4074ad, State: Running, Role: LEADER
I20260811 06:28:06.231668 27631 consensus_queue.cc:237] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } }
I20260811 06:28:06.232360 27625 sys_catalog.cc:564] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:06.234848 27632 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dae1a42f9a12430da53f18d2ae4074ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } } }
I20260811 06:28:06.235169 27632 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:06.235406 27633 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader dae1a42f9a12430da53f18d2ae4074ad. Latest consensus state: current_term: 1 leader_uuid: "dae1a42f9a12430da53f18d2ae4074ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae1a42f9a12430da53f18d2ae4074ad" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46071 } } }
I20260811 06:28:06.235721 27633 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad [sys.catalog]: This master's current role is: LEADER
W20260811 06:28:06.247081 27649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:28:06.247220 27649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:28:06.250041 27639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:06.251874 27639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:06.259845 27639 catalog_manager.cc:1347] Generated new cluster ID: 0f64b45738f840a8b48bbfddc3042147
I20260811 06:28:06.259940 27639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:06.277890 27639 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:06.279160 27639 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:06.300967 27639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: Generated new TSK 0
I20260811 06:28:06.301743 27639 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:28:06.378309 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:06.714947 27656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:06.715214 27656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:06.715382 27656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:06.722669 27656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:06.722795 27656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:06.722976 27656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:28:06.732949 27656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:06.734124 27656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:06.735437 27656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:06.748456 27665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:06.748770 27662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:06.748581 27663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:06.749598 27656 server_base.cc:1034] running on GCE node
I20260811 06:28:06.750136 27656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:06.751003 27656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:06.752225 27656 hybrid_clock.cc:648] HybridClock initialized: now 1786429686752191 us; error 45 us; skew 500 ppm
I20260811 06:28:06.752544 27656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:06 dist-test-slave-07c2 krb5kdc[27527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429686, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:06.755769 27656 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:28:06.757196 27656 webserver.cc:460] Webserver started at http://127.7.4.129:40317/ using document root <none> and password file <none>
I20260811 06:28:06.757874 27656 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:06.757977 27656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:06.758229 27656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:06.761169 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "c9e8b4d91e43465aa458cb8ebae3c549"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "61df2e3652f052f9b9e6f941f7c7994b"
server_key_iv: "23072005823c21f7737dea4398d7a348"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.761955 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "c9e8b4d91e43465aa458cb8ebae3c549"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "61df2e3652f052f9b9e6f941f7c7994b"
server_key_iv: "23072005823c21f7737dea4398d7a348"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.766750 27656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 06:28:06.770061 27673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:06.771277 27656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:28:06.771481 27656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "c9e8b4d91e43465aa458cb8ebae3c549"
format_stamp: "Formatted at 2026-08-11 06:28:06 on dist-test-slave-07c2"
server_key: "61df2e3652f052f9b9e6f941f7c7994b"
server_key_iv: "23072005823c21f7737dea4398d7a348"
server_key_version: "encryptionkey@0"
I20260811 06:28:06.771632 27656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:06.782379 27656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:06.786468 27656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:06.786711 27656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:06.787614 27656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:06.788934 27656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:06.789007 27656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:06.789088 27656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:06.789132 27656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:06.823065 27656 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:45999
I20260811 06:28:06.823123 27813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:45999 every 8 connection(s)
I20260811 06:28:06.824414 27656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:28:06.828931 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27656
I20260811 06:28:06.829118 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:28:06.829741 7186 external_mini_cluster.cc:1428] Setting key 4bf5041c78da78d393ccd36bddedb361
Aug 11 06:28:06 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429686, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:06.851362 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:06.826917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:46887 (local address 127.7.4.190:46071)
0811 06:28:06.827439 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:06.827446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:06.828366 (+ 920us) server_negotiation.cc:407] Connection header received
0811 06:28:06.830826 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:06.830833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:06.830909 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:06.831002 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:06.833050 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.833915 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.836380 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.836796 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.836897 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:06.845469 (+ 8572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:06.845474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:06.845478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:06.845509 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:06.848223 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:06.848997 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:06.849001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:06.849003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:06.849123 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:06.849565 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:06.849570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:06.849575 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:06.849857 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:06.850013 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:06.850738 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 06:28:06.850971 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":234,"threads_started":1}
I20260811 06:28:06.852334 27818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:06.827467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46071 (local address 127.7.4.129:46887)
0811 06:28:06.828191 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:06.828231 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:06.830578 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:06.831282 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:06.831312 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:06.831982 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:06.832856 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:06.832884 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.834070 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.834078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:06.836221 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:06.836246 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:06.837030 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:06.837039 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:06.837351 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:06.842417 (+ 5066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:06.842452 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:06.845224 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:06.848374 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:06.848384 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:06.848401 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:06.848854 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:06.849238 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:06.849243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:06.849247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:06.849441 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:06.850009 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:06.850017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:06.850347 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 06:28:06.850610 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 06:28:06.851157 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":277,"threads_started":1}
I20260811 06:28:06.854040 27814 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46071
I20260811 06:28:06.854440 27814 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:06.855370 27814 heartbeater.cc:511] Master 127.7.4.190:46071 requested a full tablet report, sending...
I20260811 06:28:06.857991 27584 ts_manager.cc:194] Registered new tserver with Master: c9e8b4d91e43465aa458cb8ebae3c549 (127.7.4.129:45999)
I20260811 06:28:06.860303 27584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:46887
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:28:06.906315 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:07.246592 27821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:07.246872 27821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:07.247041 27821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:07.254410 27821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:07.254546 27821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:07.254738 27821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:28:07.264395 27821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:07.265594 27821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:07.266866 27821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:07.280189 27827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:07.280328 27826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:07.280388 27829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:07.281075 27821 server_base.cc:1034] running on GCE node
I20260811 06:28:07.281715 27821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:07.282660 27821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:07.283901 27821 hybrid_clock.cc:648] HybridClock initialized: now 1786429687283862 us; error 51 us; skew 500 ppm
I20260811 06:28:07.284210 27821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:07 dist-test-slave-07c2 krb5kdc[27527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:07.287272 27821 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:28:07.288817 27821 webserver.cc:460] Webserver started at http://127.7.4.130:36275/ using document root <none> and password file <none>
I20260811 06:28:07.289498 27821 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:07.289656 27821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:07.289966 27821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:07.292867 27821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "ecd583dfba4d4e4b8e16ab6f226a838b"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "366adf2162f149528770c4135fa1ecb8"
server_key_iv: "1c75c12159958ed04b8058622bd3584b"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.293648 27821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "ecd583dfba4d4e4b8e16ab6f226a838b"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "366adf2162f149528770c4135fa1ecb8"
server_key_iv: "1c75c12159958ed04b8058622bd3584b"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.298626 27821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:28:07.302176 27837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.303478 27821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:07.303643 27821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "ecd583dfba4d4e4b8e16ab6f226a838b"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "366adf2162f149528770c4135fa1ecb8"
server_key_iv: "1c75c12159958ed04b8058622bd3584b"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.303846 27821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:07.330976 27821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:07.335747 27821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:07.336035 27821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:07.337010 27821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:07.338286 27821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:07.338364 27821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.338533 27821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:07.338584 27821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.371559 27821 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:41739
I20260811 06:28:07.371637 27970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:41739 every 8 connection(s)
I20260811 06:28:07.372850 27821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:28:07.375639 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27821
I20260811 06:28:07.375854 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:28:07.376423 7186 external_mini_cluster.cc:1428] Setting key 1c40f50b48db6378ad5aee39758bc692
Aug 11 06:28:07 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:07.393187 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.375640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:50125 (local address 127.7.4.190:46071)
0811 06:28:07.376224 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:07.376230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:07.377355 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 06:28:07.378999 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:07.379005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:07.379091 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:07.379216 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:07.380757 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.381499 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.383033 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.383532 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.383670 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:07.387385 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:07.387390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:07.387393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:07.387425 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:07.390103 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.390807 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.390814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.390818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.390953 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.391439 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.391445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.391448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.391729 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:07.391891 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:07.392551 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 06:28:07.392786 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":251,"threads_started":1}
I20260811 06:28:07.394078 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.376139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46071 (local address 127.7.4.130:50125)
0811 06:28:07.376964 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:07.377011 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:07.378745 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:07.379469 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:07.379476 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:07.379966 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:07.380593 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:07.380614 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.381650 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.381656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:07.382870 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:07.382883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.383752 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.383761 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:07.383989 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:07.384721 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:07.384755 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:07.387135 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:07.390255 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:07.390263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:07.390277 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:07.390691 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:07.391067 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:07.391072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:07.391075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:07.391275 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:07.391865 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:07.391873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:07.392199 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 06:28:07.392454 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 06:28:07.392966 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":351,"threads_started":1}
I20260811 06:28:07.395627 27971 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46071
I20260811 06:28:07.395996 27971 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:07.396885 27971 heartbeater.cc:511] Master 127.7.4.190:46071 requested a full tablet report, sending...
I20260811 06:28:07.398572 27584 ts_manager.cc:194] Registered new tserver with Master: ecd583dfba4d4e4b8e16ab6f226a838b (127.7.4.130:41739)
I20260811 06:28:07.399860 27584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:50125
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:28:07.448019 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:41973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:07.785692 27977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:07.785966 27977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:07.786139 27977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:07.793241 27977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:07.793365 27977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:07.793541 27977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:07.803752 27977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:07.805003 27977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:07.806295 27977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:07.819136 27987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:07.819216 27985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:07.819234 27984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:07.820046 27977 server_base.cc:1034] running on GCE node
I20260811 06:28:07.820637 27977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:07.821532 27977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:07.822752 27977 hybrid_clock.cc:648] HybridClock initialized: now 1786429687822713 us; error 52 us; skew 500 ppm
I20260811 06:28:07.823040 27977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:07 dist-test-slave-07c2 krb5kdc[27527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:07.826472 27977 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:28:07.828148 27977 webserver.cc:460] Webserver started at http://127.7.4.131:35477/ using document root <none> and password file <none>
I20260811 06:28:07.828799 27977 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:07.828902 27977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:07.829144 27977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:07.831904 27977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "500bbc4c90df4db0a7fd6b507ef65d73"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "024600beed6fd56bd2828515305ff459"
server_key_iv: "2769f57d590e79a6086c17455be312fc"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.832662 27977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "500bbc4c90df4db0a7fd6b507ef65d73"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "024600beed6fd56bd2828515305ff459"
server_key_iv: "2769f57d590e79a6086c17455be312fc"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.837762 27977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:28:07.841243 27995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.842499 27977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:28:07.842669 27977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "500bbc4c90df4db0a7fd6b507ef65d73"
format_stamp: "Formatted at 2026-08-11 06:28:07 on dist-test-slave-07c2"
server_key: "024600beed6fd56bd2828515305ff459"
server_key_iv: "2769f57d590e79a6086c17455be312fc"
server_key_version: "encryptionkey@0"
I20260811 06:28:07.842872 27977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:07.864190 27814 heartbeater.cc:503] Master 127.7.4.190:46071 was elected leader, sending a full tablet report...
I20260811 06:28:07.877400 27977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:07.881254 27977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:07.881574 27977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:07.882505 27977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:07.883862 27977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:07.883936 27977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.884035 27977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:07.884079 27977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:07.917740 27977 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:39509
I20260811 06:28:07.917833 28126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:39509 every 8 connection(s)
I20260811 06:28:07.919008 27977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:07.927831 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 27977
I20260811 06:28:07.928020 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:07.928467 7186 external_mini_cluster.cc:1428] Setting key 286c2a94c745ff41f8a8af3f1a75de73
Aug 11 06:28:07 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429687, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:07.939033 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.921652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39561 (local address 127.7.4.190:46071)
0811 06:28:07.922113 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:07.922118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:07.923075 (+ 957us) server_negotiation.cc:407] Connection header received
0811 06:28:07.924957 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:07.924963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:07.925049 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:07.925191 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:07.926629 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.927561 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.929297 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.929886 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.930009 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:07.933858 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:07.933863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:07.933867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:07.933905 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:07.936476 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.937143 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.937146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.937148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.937264 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.937628 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.937631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.937633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.937885 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:07.937995 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:07.938583 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 06:28:07.938740 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":226,"threads_started":1}
I20260811 06:28:07.940026 28131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.922242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46071 (local address 127.7.4.131:39561)
0811 06:28:07.922915 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:07.922951 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:07.924677 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:07.925431 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:07.925439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:07.925937 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:07.926459 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:07.926474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.927723 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.927727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:07.929161 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:07.929175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.930151 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.930159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:07.930440 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:07.931194 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:07.931231 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:07.933672 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:07.936603 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:07.936611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:07.936621 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:07.937043 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:07.937359 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:07.937362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:07.937364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:07.937536 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:07.937989 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:07.937994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:07.938279 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 06:28:07.938496 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 06:28:07.938862 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":280,"threads_started":1}
I20260811 06:28:07.941402 28127 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46071
I20260811 06:28:07.941696 28127 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:07.942471 28127 heartbeater.cc:511] Master 127.7.4.190:46071 requested a full tablet report, sending...
I20260811 06:28:07.944012 27584 ts_manager.cc:194] Registered new tserver with Master: 500bbc4c90df4db0a7fd6b507ef65d73 (127.7.4.131:39509)
I20260811 06:28:07.944810 27584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:39561
I20260811 06:28:07.956080 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:07.973846 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.960336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.7.4.190:46071)
0811 06:28:07.960547 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:07.960556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:07.961039 (+ 483us) server_negotiation.cc:407] Connection header received
0811 06:28:07.961238 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:07.961243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:07.961324 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:07.961428 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:07.962954 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.963710 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.965003 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:07.965375 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:07.965496 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:07.968432 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:07.968439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:07.968444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:07.968483 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:07.971051 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.971843 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.971855 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.971858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.971974 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:07.972443 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:07.972455 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:07.972458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:07.972731 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:07.973016 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:07.973368 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 06:28:07.973566 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:07.977504 27584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:07.984161 27584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:08.018025 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.995732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.7.4.131:39509)
0811 06:28:08.008213 (+ 12481us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.008226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.008267 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:28:08.008401 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.008412 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.008745 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.009723 (+ 978us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:08.010039 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.011683 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.015395 (+ 3712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.016757 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.016887 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.017134 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.017473 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.017634 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12254,"thread_start_us":12092,"threads_started":1}
I20260811 06:28:08.018096 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.995706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:39509 (local address 127.0.0.1:58498)
0811 06:28:08.007248 (+ 11542us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:08.007279 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:08.007413 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:08.009047 (+ 1634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:08.009058 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:08.009103 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:08.009656 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.009673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.011815 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.011820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.015183 (+ 3363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.015199 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.016958 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.016991 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.017207 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.017260 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 06:28:08.017399 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:28:08.017650 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11432,"mutex_wait_us":384,"thread_start_us":11462,"threads_started":1}
I20260811 06:28:08.018194 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.996300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.7.4.130:41739)
0811 06:28:08.009766 (+ 13466us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.009774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.009799 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:28:08.009888 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.009895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.010100 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.010287 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:08.010947 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.012412 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.015316 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.016623 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.016750 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.016995 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.017666 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.017855 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13268,"thread_start_us":13107,"threads_started":1}
I20260811 06:28:08.018757 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.995705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:41739 (local address 127.0.0.1:35960)
0811 06:28:08.006391 (+ 10686us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:08.006444 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:08.006641 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:08.010271 (+ 3630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:08.010275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:08.010300 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:08.010728 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.010742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.012569 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.012574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.015128 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.015144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.017131 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.017138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.017276 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.017301 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:28:08.017399 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:28:08.018397 (+ 998us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10350,"mutex_wait_us":860,"thread_start_us":11275,"threads_started":1}
I20260811 06:28:08.019007 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.997119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.7.4.129:45999)
0811 06:28:08.008609 (+ 11490us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.008621 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.009854 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 06:28:08.009960 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.009967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.010195 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.010405 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:08.011132 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.012530 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.016046 (+ 3516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.017423 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.017546 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.017751 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.018506 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.018663 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11272,"thread_start_us":11075,"threads_started":1}
I20260811 06:28:08.019388 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:07.996372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:45999 (local address 127.0.0.1:33050)
0811 06:28:08.009062 (+ 12690us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:08.009084 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:08.009215 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:08.010576 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:08.010585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:08.010608 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:08.010981 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.010990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.012708 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.012717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.015875 (+ 3158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.015887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.017902 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.017908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.018100 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.018140 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:28:08.018262 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:28:08.018366 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12574,"mutex_wait_us":47,"thread_start_us":12418,"threads_started":1}
I20260811 06:28:08.022468 27898 tablet_service.cc:1467] Processing CreateTablet for tablet 51195a82a64e4acc948fffcab09fcc90 (DEFAULT_TABLE table=test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:08.022534 28052 tablet_service.cc:1467] Processing CreateTablet for tablet 51195a82a64e4acc948fffcab09fcc90 (DEFAULT_TABLE table=test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:08.023056 27732 tablet_service.cc:1467] Processing CreateTablet for tablet 51195a82a64e4acc948fffcab09fcc90 (DEFAULT_TABLE table=test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:08.023838 27898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51195a82a64e4acc948fffcab09fcc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:08.024071 28052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51195a82a64e4acc948fffcab09fcc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:08.024394 27732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51195a82a64e4acc948fffcab09fcc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:08.035243 28148 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Bootstrap starting.
I20260811 06:28:08.038516 28150 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Bootstrap starting.
I20260811 06:28:08.038614 28148 tablet_bootstrap.cc:654] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:08.040241 28149 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Bootstrap starting.
I20260811 06:28:08.040920 28148 log.cc:824] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:08.042429 28150 tablet_bootstrap.cc:654] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:08.043329 28149 tablet_bootstrap.cc:654] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:08.043684 28150 log.cc:824] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:08.044438 28149 log.cc:824] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:08.044499 28148 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: No bootstrap required, opened a new log
I20260811 06:28:08.044761 28148 ts_tablet_manager.cc:1397] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:28:08.046866 28150 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: No bootstrap required, opened a new log
I20260811 06:28:08.047142 28150 ts_tablet_manager.cc:1397] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:28:08.048321 28149 tablet_bootstrap.cc:492] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: No bootstrap required, opened a new log
I20260811 06:28:08.048591 28149 ts_tablet_manager.cc:1397] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 06:28:08.055675 28149 raft_consensus.cc:348] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.056030 28149 raft_consensus.cc:374] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:08.056175 28149 raft_consensus.cc:729] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e8b4d91e43465aa458cb8ebae3c549, State: Initialized, Role: FOLLOWER
I20260811 06:28:08.055936 28148 raft_consensus.cc:348] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.056356 28148 raft_consensus.cc:374] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:08.056456 28148 raft_consensus.cc:729] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd583dfba4d4e4b8e16ab6f226a838b, State: Initialized, Role: FOLLOWER
I20260811 06:28:08.056994 28149 consensus_queue.cc:260] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.057087 28148 consensus_queue.cc:260] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.057560 28150 raft_consensus.cc:348] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.058185 28150 raft_consensus.cc:374] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:08.058490 28150 raft_consensus.cc:729] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500bbc4c90df4db0a7fd6b507ef65d73, State: Initialized, Role: FOLLOWER
I20260811 06:28:08.058780 28149 ts_tablet_manager.cc:1428] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 06:28:08.059271 28150 consensus_queue.cc:260] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.064419 27971 heartbeater.cc:503] Master 127.7.4.190:46071 was elected leader, sending a full tablet report...
I20260811 06:28:08.065673 28148 ts_tablet_manager.cc:1428] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Time spent starting tablet: real 0.021s user 0.010s sys 0.010s
I20260811 06:28:08.066633 28127 heartbeater.cc:503] Master 127.7.4.190:46071 was elected leader, sending a full tablet report...
I20260811 06:28:08.067265 28150 ts_tablet_manager.cc:1428] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
W20260811 06:28:08.076550 27815 tablet.cc:2369] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:28:08.124943 27972 tablet.cc:2369] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:28:08.170725 28129 tablet.cc:2369] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:08.394936 28154 raft_consensus.cc:482] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:08.395207 28154 raft_consensus.cc:504] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.396907 28154 leader_election.cc:288] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500bbc4c90df4db0a7fd6b507ef65d73 (127.7.4.131:39509), ecd583dfba4d4e4b8e16ab6f226a838b (127.7.4.130:41739)
I20260811 06:28:08.403481 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.397310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:53313 (local address 127.7.4.131:39509)
0811 06:28:08.397703 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.397713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.398097 (+ 384us) server_negotiation.cc:407] Connection header received
0811 06:28:08.398352 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.398357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.398417 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.398513 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:08.399077 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.399865 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.401747 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.402518 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.402667 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.402784 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.403070 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.403190 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260811 06:28:08.403666 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.397213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:39509 (local address 127.7.4.129:53313)
0811 06:28:08.397982 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:08.397998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:08.398188 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:08.398589 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:08.398594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:08.398621 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:08.398927 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.398935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.400001 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.400009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.401586 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.401599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.402728 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.402734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.402903 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.402928 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:28:08.403043 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:28:08.403171 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":41,"thread_start_us":457,"threads_started":1}
I20260811 06:28:08.403687 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.397446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:56155 (local address 127.7.4.130:41739)
0811 06:28:08.397662 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.397670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.398504 (+ 834us) server_negotiation.cc:407] Connection header received
0811 06:28:08.398714 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.398718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.398773 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.398868 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:08.399524 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.400202 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.401976 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.402738 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.402896 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.402995 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.403274 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.403429 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 06:28:08.403831 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.397345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:41739 (local address 127.7.4.129:56155)
0811 06:28:08.398396 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:08.398415 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:08.398543 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:08.398892 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:08.398900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:08.398930 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:08.399353 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.399369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.400350 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.400356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.401812 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:08.401825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.402960 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.402966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:08.403114 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.403138 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:28:08.403244 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:28:08.403418 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":22,"thread_start_us":641,"threads_started":1}
I20260811 06:28:08.404750 28075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51195a82a64e4acc948fffcab09fcc90" candidate_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" is_pre_election: true
I20260811 06:28:08.404806 27921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51195a82a64e4acc948fffcab09fcc90" candidate_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" is_pre_election: true
I20260811 06:28:08.405230 28075 raft_consensus.cc:2454] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9e8b4d91e43465aa458cb8ebae3c549 in term 0.
I20260811 06:28:08.405301 27921 raft_consensus.cc:2454] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9e8b4d91e43465aa458cb8ebae3c549 in term 0.
I20260811 06:28:08.405956 27678 leader_election.cc:302] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500bbc4c90df4db0a7fd6b507ef65d73, c9e8b4d91e43465aa458cb8ebae3c549; no voters:
I20260811 06:28:08.406365 28154 raft_consensus.cc:2781] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:08.406512 28154 raft_consensus.cc:482] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:08.406631 28154 raft_consensus.cc:3037] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:08.408924 28154 raft_consensus.cc:504] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.409523 28154 leader_election.cc:288] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [CANDIDATE]: Term 1 election: Requested vote from peers 500bbc4c90df4db0a7fd6b507ef65d73 (127.7.4.131:39509), ecd583dfba4d4e4b8e16ab6f226a838b (127.7.4.130:41739)
I20260811 06:28:08.409953 28075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51195a82a64e4acc948fffcab09fcc90" candidate_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500bbc4c90df4db0a7fd6b507ef65d73"
I20260811 06:28:08.410012 27921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51195a82a64e4acc948fffcab09fcc90" candidate_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b"
I20260811 06:28:08.410132 28075 raft_consensus.cc:3037] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:08.410245 27921 raft_consensus.cc:3037] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:08.412438 28075 raft_consensus.cc:2454] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9e8b4d91e43465aa458cb8ebae3c549 in term 1.
I20260811 06:28:08.412590 27921 raft_consensus.cc:2454] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9e8b4d91e43465aa458cb8ebae3c549 in term 1.
I20260811 06:28:08.412859 27678 leader_election.cc:302] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 500bbc4c90df4db0a7fd6b507ef65d73, c9e8b4d91e43465aa458cb8ebae3c549; no voters:
I20260811 06:28:08.413129 28154 raft_consensus.cc:2781] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:08.413565 28154 raft_consensus.cc:686] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 1 LEADER]: Becoming Leader. State: Replica: c9e8b4d91e43465aa458cb8ebae3c549, State: Running, Role: LEADER
I20260811 06:28:08.414026 28154 consensus_queue.cc:237] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } }
I20260811 06:28:08.418776 27583 catalog_manager.cc:5495] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9e8b4d91e43465aa458cb8ebae3c549 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9e8b4d91e43465aa458cb8ebae3c549" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 45999 } health_report { overall_health: HEALTHY } } }
I20260811 06:28:08.476032 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.470577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.7.4.129:45999)
0811 06:28:08.470825 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.470830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.470882 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:28:08.471073 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.471077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.471168 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.471310 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:08.471716 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.472457 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.473772 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.474198 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.474285 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.474824 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:08.475367 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:08.475493 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.475639 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.475686 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:28:08.483943 28075 raft_consensus.cc:1261] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 1 FOLLOWER]: Refusing update from remote peer c9e8b4d91e43465aa458cb8ebae3c549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:08.483945 27921 raft_consensus.cc:1261] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 1 FOLLOWER]: Refusing update from remote peer c9e8b4d91e43465aa458cb8ebae3c549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:08.484920 28159 consensus_queue.cc:1035] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd583dfba4d4e4b8e16ab6f226a838b" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:08.485285 28154 consensus_queue.cc:1035] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "500bbc4c90df4db0a7fd6b507ef65d73" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:08.496297 28163 mvcc.cc:205] Tried to move back new op lower bound from 7317216004020420608 to 7317216003747606528. Current Snapshot: MvccSnapshot[applied={T|T < 7317216004020420608}]
I20260811 06:28:08.509933 27584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:08.510191 27584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:08.513489 27584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dae1a42f9a12430da53f18d2ae4074ad: Sending DeleteTablet for 3 replicas of tablet 51195a82a64e4acc948fffcab09fcc90
I20260811 06:28:08.514673 27898 tablet_service.cc:1514] Processing DeleteTablet for tablet 51195a82a64e4acc948fffcab09fcc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:35960
I20260811 06:28:08.514672 27732 tablet_service.cc:1514] Processing DeleteTablet for tablet 51195a82a64e4acc948fffcab09fcc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:33050
I20260811 06:28:08.514670 28052 tablet_service.cc:1514] Processing DeleteTablet for tablet 51195a82a64e4acc948fffcab09fcc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:08 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:58498
I20260811 06:28:08.521243 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27656
I20260811 06:28:08.527935 28175 tablet_replica.cc:330] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: stopping tablet replica
I20260811 06:28:08.528393 28175 raft_consensus.cc:2229] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:08.528927 28175 raft_consensus.cc:2258] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:08.529742 28176 tablet_replica.cc:330] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: stopping tablet replica
I20260811 06:28:08.530156 28176 raft_consensus.cc:2229] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:08.530789 28176 raft_consensus.cc:2258] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:08.532960 28175 ts_tablet_manager.cc:1905] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:08.534974 28178 tablet_replica.cc:330] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: stopping tablet replica
I20260811 06:28:08.535420 28178 raft_consensus.cc:2229] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:08.536604 28176 ts_tablet_manager.cc:1905] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:08.540496 28175 ts_tablet_manager.cc:1918] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:08.540731 28175 log.cc:1196] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/51195a82a64e4acc948fffcab09fcc90
I20260811 06:28:08.540819 28178 raft_consensus.cc:2258] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:08.541419 28175 ts_tablet_manager.cc:1939] T 51195a82a64e4acc948fffcab09fcc90 P ecd583dfba4d4e4b8e16ab6f226a838b: Deleting consensus metadata
Aug 11 06:28:08 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:28:08.543236 28178 ts_tablet_manager.cc:1905] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:08.544297 28176 ts_tablet_manager.cc:1918] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:08.544510 28176 log.cc:1196] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/51195a82a64e4acc948fffcab09fcc90
I20260811 06:28:08.545154 28176 ts_tablet_manager.cc:1939] T 51195a82a64e4acc948fffcab09fcc90 P c9e8b4d91e43465aa458cb8ebae3c549: Deleting consensus metadata
I20260811 06:28:08.545697 27564 catalog_manager.cc:4841] TS ecd583dfba4d4e4b8e16ab6f226a838b (127.7.4.130:41739): tablet 51195a82a64e4acc948fffcab09fcc90 (table test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]) successfully deleted
I20260811 06:28:08.546876 27568 catalog_manager.cc:4841] TS c9e8b4d91e43465aa458cb8ebae3c549 (127.7.4.129:45999): tablet 51195a82a64e4acc948fffcab09fcc90 (table test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]) successfully deleted
I20260811 06:28:08.550694 28178 ts_tablet_manager.cc:1918] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:08.550946 28178 log.cc:1196] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/51195a82a64e4acc948fffcab09fcc90
I20260811 06:28:08.551406 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.525824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.7.4.129:45999)
0811 06:28:08.526041 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.526049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.526072 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:28:08.526137 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.526145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.526238 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.526380 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:08.534532 (+ 8152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.535802 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.537757 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.538515 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.538620 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.544604 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:08.544619 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:08.544636 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:08.544701 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:08.548501 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.549213 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.549220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.549223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.549374 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.549780 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.549784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.549786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.550116 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:08.550452 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.550763 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.551003 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:28:08.552186 28178 ts_tablet_manager.cc:1939] T 51195a82a64e4acc948fffcab09fcc90 P 500bbc4c90df4db0a7fd6b507ef65d73: Deleting consensus metadata
I20260811 06:28:08.552218 27808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:08.554034 27567 catalog_manager.cc:4841] TS 500bbc4c90df4db0a7fd6b507ef65d73 (127.7.4.131:39509): tablet 51195a82a64e4acc948fffcab09fcc90 (table test-table [id=928e9e35f89e49b39ce7dfd26333d7b9]) successfully deleted
I20260811 06:28:08.660791 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27656
I20260811 06:28:08.691254 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27821
Aug 11 06:28:08 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:28:08.706703 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.692184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.7.4.130:41739)
0811 06:28:08.692365 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.692372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.692396 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:28:08.692631 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.692644 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.692713 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.692819 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:08.694104 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.694816 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.696309 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.696713 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.696852 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.701013 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:08.701027 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:08.701043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:08.701082 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:08.703928 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.704633 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.704640 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.704642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.704765 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.705148 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.705154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.705157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.705560 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:08.705870 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.706156 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.706373 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:28:08.707428 27966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:08.801862 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27821
I20260811 06:28:08.823714 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27977
Aug 11 06:28:08 dist-test-slave-07c2 krb5kdc[27527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:28:08.838325 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:08.824518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.7.4.131:39509)
0811 06:28:08.824687 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:08.824691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:08.824710 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:28:08.824875 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:08.824879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:08.824945 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:08.825031 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:08.826276 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.826981 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.828347 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:08.828750 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:08.828851 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:08.833184 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:08.833198 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:08.833214 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:08.833249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:08.835933 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.836522 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.836528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.836530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.836659 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:08.837083 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:08.837086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:08.837088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:08.837426 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:08.837633 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:08.837906 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 06:28:08.838059 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:28:08.838960 28122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:08.925388 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27977
I20260811 06:28:08.947379 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 27544
I20260811 06:28:08.948462 27618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:09.060817 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 27544
2026-08-11T06:28:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4741 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T06:28:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:09Z Disabled control of system clock
I20260811 06:28:09.157188 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41405
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:09.489904 28187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:09.490180 28187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:09.490257 28187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:09.497435 28187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:09.497539 28187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:09.497586 28187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:09.497642 28187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:09.497685 28187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:09.506440 28187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:41405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:09.507627 28187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:09.509344 28187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:09.520797 28193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:09.520718 28196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:09.521251 28194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:09.521624 28187 server_base.cc:1034] running on GCE node
I20260811 06:28:09.522348 28187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:09.523756 28187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:09.524983 28187 hybrid_clock.cc:648] HybridClock initialized: now 1786429689524957 us; error 38 us; skew 500 ppm
I20260811 06:28:09.525307 28187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:09.527606 28187 webserver.cc:460] Webserver started at http://127.7.4.190:42321/ using document root <none> and password file <none>
I20260811 06:28:09.528282 28187 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:09.528414 28187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:09.528764 28187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:09.531597 28187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "fdc3651ec34841e79a106b672250b22d"
format_stamp: "Formatted at 2026-08-11 06:28:09 on dist-test-slave-07c2"
server_key: "0bcfe56025081c7f0694e962a1ddb286"
server_key_iv: "a077683f7809b871a6613dd483135e30"
server_key_version: "encryptionkey@0"
I20260811 06:28:09.532387 28187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "fdc3651ec34841e79a106b672250b22d"
format_stamp: "Formatted at 2026-08-11 06:28:09 on dist-test-slave-07c2"
server_key: "0bcfe56025081c7f0694e962a1ddb286"
server_key_iv: "a077683f7809b871a6613dd483135e30"
server_key_version: "encryptionkey@0"
I20260811 06:28:09.537279 28187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:28:09.540669 28202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:09.541910 28187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:28:09.542245 28187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "fdc3651ec34841e79a106b672250b22d"
format_stamp: "Formatted at 2026-08-11 06:28:09 on dist-test-slave-07c2"
server_key: "0bcfe56025081c7f0694e962a1ddb286"
server_key_iv: "a077683f7809b871a6613dd483135e30"
server_key_version: "encryptionkey@0"
I20260811 06:28:09.542479 28187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:09.561548 28187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:09.562335 28187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:09.562618 28187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:09.583250 28187 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:41405
I20260811 06:28:09.583323 28259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:41405 every 8 connection(s)
I20260811 06:28:09.584578 28187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:09.588112 28260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:09.593884 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28187
I20260811 06:28:09.594069 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:09.594521 7186 external_mini_cluster.cc:1428] Setting key 21e5cf4a0f2236552cbec3488bf798ac
I20260811 06:28:09.598341 28260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: Bootstrap starting.
I20260811 06:28:09.601547 28260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:09.602838 28260 log.cc:824] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:09.605947 28260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: No bootstrap required, opened a new log
I20260811 06:28:09.608328 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:09.596278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.7.4.190:41405)
0811 06:28:09.597218 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:09.597228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:09.597267 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:28:09.598954 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:09.598973 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:09.599452 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:09.599772 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:09.600387 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:09.601607 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:09.603669 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:09.604515 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:09.604641 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:09.605846 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:09.605853 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:09.605864 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:09.606441 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:09.606508 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:09.606532 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:09.606868 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:09.607130 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 06:28:09.607462 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":351,"threads_started":1}
I20260811 06:28:09.613557 28260 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } }
I20260811 06:28:09.613909 28260 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:09.614014 28260 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc3651ec34841e79a106b672250b22d, State: Initialized, Role: FOLLOWER
I20260811 06:28:09.614519 28260 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } }
I20260811 06:28:09.614769 28260 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:09.614912 28260 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:09.615084 28260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:09.617506 28260 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } }
I20260811 06:28:09.617977 28260 leader_election.cc:302] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc3651ec34841e79a106b672250b22d; no voters:
I20260811 06:28:09.618546 28260 leader_election.cc:288] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:09.618714 28265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:09.619488 28265 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [term 1 LEADER]: Becoming Leader. State: Replica: fdc3651ec34841e79a106b672250b22d, State: Running, Role: LEADER
I20260811 06:28:09.620003 28265 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } }
I20260811 06:28:09.620577 28260 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:09.622558 28267 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc3651ec34841e79a106b672250b22d. Latest consensus state: current_term: 1 leader_uuid: "fdc3651ec34841e79a106b672250b22d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } } }
I20260811 06:28:09.622965 28267 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:09.622869 28266 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc3651ec34841e79a106b672250b22d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc3651ec34841e79a106b672250b22d" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41405 } } }
I20260811 06:28:09.623162 28266 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:09.624558 28271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:09.630275 28271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:09.638845 28271 catalog_manager.cc:1347] Generated new cluster ID: bf7a632098214e75abfcc3a24ddc57bd
I20260811 06:28:09.638950 28271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:09.660586 28271 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:09.662387 28271 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:09.681591 28271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: Generated new TSK 0
I20260811 06:28:09.682554 28271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:09.694278 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--builtin_ntp_servers=127.7.4.148:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:10.031764 28285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:10.032059 28285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:10.032215 28285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:10.039333 28285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:10.039453 28285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:10.039623 28285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:28:10.048907 28285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:10.050428 28285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:10.052281 28285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:10.065475 28292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.065521 28295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.065919 28291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:10.066372 28285 server_base.cc:1034] running on GCE node
I20260811 06:28:10.066959 28285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:10.067884 28285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:10.069110 28285 hybrid_clock.cc:648] HybridClock initialized: now 1786429690069071 us; error 50 us; skew 500 ppm
I20260811 06:28:10.069415 28285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:10.071524 28285 webserver.cc:460] Webserver started at http://127.7.4.129:40405/ using document root <none> and password file <none>
I20260811 06:28:10.072131 28285 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:10.072264 28285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:10.072570 28285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:10.075258 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "3fd84ba9fe1c4405be60e845c5f88ae4"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ce95c8d916a53216c37173da364bb01c"
server_key_iv: "7df86708f50d82aa4805bbc0a01c393b"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.076071 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "3fd84ba9fe1c4405be60e845c5f88ae4"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ce95c8d916a53216c37173da364bb01c"
server_key_iv: "7df86708f50d82aa4805bbc0a01c393b"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.080806 28285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:28:10.083921 28301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.085085 28285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:28:10.085248 28285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "3fd84ba9fe1c4405be60e845c5f88ae4"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ce95c8d916a53216c37173da364bb01c"
server_key_iv: "7df86708f50d82aa4805bbc0a01c393b"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.085448 28285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:10.106766 28285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:10.107609 28285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:10.107882 28285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:10.108803 28285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:10.110262 28285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:10.110381 28285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.110509 28285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:10.110596 28285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.144143 28285 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:34177
I20260811 06:28:10.144228 28433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:34177 every 8 connection(s)
I20260811 06:28:10.145574 28285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:28:10.147512 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28285
I20260811 06:28:10.147645 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:28:10.148031 7186 external_mini_cluster.cc:1428] Setting key e4bfe2f33c8f183ce95b59f01c619a36
I20260811 06:28:10.150904 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--builtin_ntp_servers=127.7.4.148:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:10.160337 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:10.149627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:59489 (local address 127.7.4.190:41405)
0811 06:28:10.150303 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:10.150310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:10.151535 (+ 1225us) server_negotiation.cc:407] Connection header received
0811 06:28:10.153277 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:10.153285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:10.153373 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:10.153492 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:10.154505 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.155718 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.157238 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.157933 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.158072 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:10.159377 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:10.159385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:10.159392 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:10.159418 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:10.159472 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:10.159491 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:10.159586 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:10.159806 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 06:28:10.159925 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":446,"threads_started":1}
I20260811 06:28:10.161015 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:10.150323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:41405 (local address 127.7.4.129:59489)
0811 06:28:10.151366 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:10.151421 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:10.153003 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:10.153671 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:10.153679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:10.153763 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:10.154317 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:10.154333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.155876 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.155880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:10.157062 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:10.157073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.158133 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.158138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:10.158340 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:10.159044 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:10.159047 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:10.159082 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:10.159163 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:10.159594 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:10.159603 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:10.159608 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:28:10.159746 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 06:28:10.160016 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":345,"threads_started":1}
I20260811 06:28:10.162508 28434 heartbeater.cc:347] Connected to a master server at 127.7.4.190:41405
I20260811 06:28:10.162807 28434 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:10.163564 28434 heartbeater.cc:511] Master 127.7.4.190:41405 requested a full tablet report, sending...
I20260811 06:28:10.165510 28220 ts_manager.cc:194] Registered new tserver with Master: 3fd84ba9fe1c4405be60e845c5f88ae4 (127.7.4.129:34177)
I20260811 06:28:10.166991 28220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.129:59489
W20260811 06:28:10.488245 28440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:10.488507 28440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:10.488734 28440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:10.495728 28440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:10.495810 28440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:10.496006 28440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:28:10.504632 28440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:10.505796 28440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:10.507567 28440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:10.520148 28448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.520233 28449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.520166 28451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:10.524616 28440 server_base.cc:1034] running on GCE node
I20260811 06:28:10.525174 28440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:10.526119 28440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:10.527371 28440 hybrid_clock.cc:648] HybridClock initialized: now 1786429690527329 us; error 33 us; skew 500 ppm
I20260811 06:28:10.527684 28440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:10.529883 28440 webserver.cc:460] Webserver started at http://127.7.4.130:46215/ using document root <none> and password file <none>
I20260811 06:28:10.530473 28440 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:10.530557 28440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:10.530808 28440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:10.533592 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "cb1e8e1cb495482e800419c90bf03137"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "3e37bd51ba8bb4aa9b9f66ffff3fc844"
server_key_iv: "90cc7f30eada3e01adac686863a86316"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.534320 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "cb1e8e1cb495482e800419c90bf03137"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "3e37bd51ba8bb4aa9b9f66ffff3fc844"
server_key_iv: "90cc7f30eada3e01adac686863a86316"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.539057 28440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 06:28:10.542445 28458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.543848 28440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:10.544019 28440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "cb1e8e1cb495482e800419c90bf03137"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "3e37bd51ba8bb4aa9b9f66ffff3fc844"
server_key_iv: "90cc7f30eada3e01adac686863a86316"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.544236 28440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:10.554708 28440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:10.555562 28440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:10.555830 28440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:10.556756 28440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:10.558210 28440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:10.558336 28440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.558459 28440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:10.558548 28440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.596587 28440 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34381
I20260811 06:28:10.596638 28596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34381 every 8 connection(s)
I20260811 06:28:10.597843 28440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:28:10.604063 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28440
I20260811 06:28:10.604260 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:28:10.604887 7186 external_mini_cluster.cc:1428] Setting key 141d977b90a19e80b1b54cd5d515e26e
I20260811 06:28:10.607558 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--builtin_ntp_servers=127.7.4.148:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:10.609979 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:10.600428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:56315 (local address 127.7.4.190:41405)
0811 06:28:10.600605 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:10.600609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:10.601923 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 06:28:10.603636 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:10.603640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:10.603705 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:10.603795 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:10.605140 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.605787 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.607323 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.607764 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.607896 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:10.609200 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:10.609205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:10.609215 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:10.609231 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:10.609271 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:10.609283 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:10.609378 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:10.609592 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:28:10.609685 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:10.610895 28599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:10.600902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:41405 (local address 127.7.4.130:56315)
0811 06:28:10.601763 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:10.601796 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:10.603420 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:10.604053 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:10.604065 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:10.604196 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:10.604971 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:10.604996 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.605944 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.605951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:10.607124 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:10.607136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:10.607987 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:10.607994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:10.608209 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:10.608907 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:10.608910 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:10.608946 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:10.609017 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:10.609380 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:10.609395 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:10.609404 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:28:10.609577 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 06:28:10.609945 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":292,"threads_started":1}
I20260811 06:28:10.612195 28597 heartbeater.cc:347] Connected to a master server at 127.7.4.190:41405
I20260811 06:28:10.612495 28597 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:10.613189 28597 heartbeater.cc:511] Master 127.7.4.190:41405 requested a full tablet report, sending...
I20260811 06:28:10.614578 28220 ts_manager.cc:194] Registered new tserver with Master: cb1e8e1cb495482e800419c90bf03137 (127.7.4.130:34381)
I20260811 06:28:10.615772 28220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.130:56315
W20260811 06:28:10.943746 28601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:10.944049 28601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:10.944211 28601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:10.951546 28601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:10.951661 28601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:10.951833 28601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:10.960801 28601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:41405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:10.962221 28601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:10.963943 28601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:10.976501 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.976462 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:10.976709 28609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:10.977339 28601 server_base.cc:1034] running on GCE node
I20260811 06:28:10.977846 28601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:10.978683 28601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:10.979876 28601 hybrid_clock.cc:648] HybridClock initialized: now 1786429690979823 us; error 52 us; skew 500 ppm
I20260811 06:28:10.980187 28601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:10.982299 28601 webserver.cc:460] Webserver started at http://127.7.4.131:42375/ using document root <none> and password file <none>
I20260811 06:28:10.982942 28601 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:10.983057 28601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:10.983458 28601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:10.986156 28601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "a16846b4bb1b43d6886c2dec8ab19888"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ac876fb4f218bd7a44cb1dd7218aa17f"
server_key_iv: "fbd7df8e60fbe0722d8439055261111d"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.986951 28601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "a16846b4bb1b43d6886c2dec8ab19888"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ac876fb4f218bd7a44cb1dd7218aa17f"
server_key_iv: "fbd7df8e60fbe0722d8439055261111d"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.992022 28601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:28:10.995214 28619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:10.996433 28601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:28:10.996562 28601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "a16846b4bb1b43d6886c2dec8ab19888"
format_stamp: "Formatted at 2026-08-11 06:28:10 on dist-test-slave-07c2"
server_key: "ac876fb4f218bd7a44cb1dd7218aa17f"
server_key_iv: "fbd7df8e60fbe0722d8439055261111d"
server_key_version: "encryptionkey@0"
I20260811 06:28:10.996731 28601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:11.014182 28601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:11.014894 28601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:11.015116 28601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:11.015997 28601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:11.017318 28601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:11.017408 28601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:11.017485 28601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:11.017540 28601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:11.050680 28601 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:38625
I20260811 06:28:11.050762 28744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:38625 every 8 connection(s)
I20260811 06:28:11.051980 28601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:11.059727 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28601
I20260811 06:28:11.059940 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:11.060432 7186 external_mini_cluster.cc:1428] Setting key 86ad459ed83297506ee137fd0ba08b55
I20260811 06:28:11.064656 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.054341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:38589 (local address 127.7.4.190:41405)
0811 06:28:11.054521 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.054525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.055817 (+ 1292us) server_negotiation.cc:407] Connection header received
0811 06:28:11.057577 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.057583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.057676 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.057798 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:11.058822 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.059814 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.061831 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.062395 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.062507 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.064022 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:11.064026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:11.064029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:11.064046 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:11.064078 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:11.064089 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:11.064161 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.064364 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.064433 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:28:11.065547 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.054783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:41405 (local address 127.7.4.131:38589)
0811 06:28:11.055661 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.055696 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.057308 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.058032 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.058040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.058122 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:11.058651 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.058666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.060023 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.060032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.061668 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.061685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.062630 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.062637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.062957 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.063749 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:11.063752 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:11.063787 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:11.063854 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:11.064178 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:11.064186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:11.064192 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.064335 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.064589 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":293,"threads_started":1}
I20260811 06:28:11.066699 28745 heartbeater.cc:347] Connected to a master server at 127.7.4.190:41405
I20260811 06:28:11.066978 28745 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:11.067751 28745 heartbeater.cc:511] Master 127.7.4.190:41405 requested a full tablet report, sending...
I20260811 06:28:11.069010 28220 ts_manager.cc:194] Registered new tserver with Master: a16846b4bb1b43d6886c2dec8ab19888 (127.7.4.131:38625)
I20260811 06:28:11.069743 28220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.131:38589
I20260811 06:28:11.075443 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:11.085407 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.079420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.7.4.190:41405)
0811 06:28:11.079560 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.079564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.080029 (+ 465us) server_negotiation.cc:407] Connection header received
0811 06:28:11.080216 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.080221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.080319 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.080418 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:11.080907 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.081547 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.082880 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.083323 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.083462 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.084652 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:11.084665 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:11.084668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:11.084684 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:11.084711 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:11.084722 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:11.084863 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.085054 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.085107 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:28:11.089072 28220 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:11.095789 28220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:11.125844 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.106619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34381 (local address 127.0.0.1:32894)
0811 06:28:11.117777 (+ 11158us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.117814 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.117994 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.119200 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.119207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.119234 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:11.119784 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.119801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.121726 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.121734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.123281 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.123323 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.124893 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.124900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.125124 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.125171 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.125308 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.125492 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10917,"thread_start_us":13276,"threads_started":1}
I20260811 06:28:11.125914 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.106944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.7.4.130:34381)
0811 06:28:11.118313 (+ 11369us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.118330 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.118356 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:28:11.118471 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.118479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.118723 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.118942 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:11.120040 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.121597 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.123549 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.124689 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.124835 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.125087 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.125397 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.125612 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11222,"spinlock_wait_cycles":18176,"thread_start_us":11193,"threads_started":1}
I20260811 06:28:11.129171 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.107293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:38625 (local address 127.0.0.1:53032)
0811 06:28:11.120208 (+ 12915us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.120225 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.120355 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.121071 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.121081 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.121110 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:11.121576 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.121591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.123999 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.124004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.126177 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.126193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.128307 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.128314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.128491 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.128525 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.128671 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.128812 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12728,"mutex_wait_us":1,"thread_start_us":10037,"threads_started":1}
I20260811 06:28:11.129247 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.108303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53032 (local address 127.7.4.131:38625)
0811 06:28:11.118842 (+ 10539us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.118851 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.120481 (+ 1630us) server_negotiation.cc:407] Connection header received
0811 06:28:11.120588 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.120595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.120815 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.121014 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:11.121846 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.123435 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.126370 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.127786 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.127914 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.128176 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.128746 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.128917 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10371,"thread_start_us":10231,"threads_started":1}
I20260811 06:28:11.129976 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.107005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:34177 (local address 127.0.0.1:36222)
0811 06:28:11.119777 (+ 12772us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.119795 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.119952 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.120875 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.120880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.120904 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:11.121433 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.121448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.123473 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.123478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.125887 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.125902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.129135 (+ 3233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.129159 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.129366 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.129397 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.129485 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.129605 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12604,"mutex_wait_us":164,"thread_start_us":12448,"threads_started":1}
I20260811 06:28:11.130095 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.107901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.7.4.129:34177)
0811 06:28:11.119254 (+ 11353us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.119264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.120098 (+ 834us) server_negotiation.cc:407] Connection header received
0811 06:28:11.120203 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.120211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.120446 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.120623 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:11.121620 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.123244 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.126333 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.128664 (+ 2331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.128785 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.129002 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.129568 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.129738 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11196,"thread_start_us":11079,"threads_started":1}
I20260811 06:28:11.130090 28521 tablet_service.cc:1467] Processing CreateTablet for tablet 415d95676fd240d085a120471b1efda7 (DEFAULT_TABLE table=test-table [id=97b8435c68f642f89857341ea38d136f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:11.133033 28672 tablet_service.cc:1467] Processing CreateTablet for tablet 415d95676fd240d085a120471b1efda7 (DEFAULT_TABLE table=test-table [id=97b8435c68f642f89857341ea38d136f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:11.133392 28357 tablet_service.cc:1467] Processing CreateTablet for tablet 415d95676fd240d085a120471b1efda7 (DEFAULT_TABLE table=test-table [id=97b8435c68f642f89857341ea38d136f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:11.134225 28672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 415d95676fd240d085a120471b1efda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:11.134546 28357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 415d95676fd240d085a120471b1efda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:11.137228 28521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 415d95676fd240d085a120471b1efda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:11.148653 28767 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Bootstrap starting.
I20260811 06:28:11.150180 28765 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Bootstrap starting.
I20260811 06:28:11.153173 28767 tablet_bootstrap.cc:654] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:11.153622 28765 tablet_bootstrap.cc:654] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:11.154702 28767 log.cc:824] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:11.154934 28765 log.cc:824] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:11.155491 28768 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Bootstrap starting.
I20260811 06:28:11.158452 28765 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: No bootstrap required, opened a new log
I20260811 06:28:11.158759 28767 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: No bootstrap required, opened a new log
I20260811 06:28:11.158758 28765 ts_tablet_manager.cc:1397] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:28:11.159024 28768 tablet_bootstrap.cc:654] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:11.159075 28767 ts_tablet_manager.cc:1397] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 06:28:11.160075 28768 log.cc:824] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:11.162357 28768 tablet_bootstrap.cc:492] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: No bootstrap required, opened a new log
I20260811 06:28:11.162603 28768 ts_tablet_manager.cc:1397] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:28:11.169533 28767 raft_consensus.cc:348] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.169533 28765 raft_consensus.cc:348] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.169569 28768 raft_consensus.cc:348] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.169889 28434 heartbeater.cc:503] Master 127.7.4.190:41405 was elected leader, sending a full tablet report...
I20260811 06:28:11.169981 28765 raft_consensus.cc:374] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:11.169988 28768 raft_consensus.cc:374] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:11.170081 28768 raft_consensus.cc:729] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb1e8e1cb495482e800419c90bf03137, State: Initialized, Role: FOLLOWER
I20260811 06:28:11.170089 28765 raft_consensus.cc:729] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd84ba9fe1c4405be60e845c5f88ae4, State: Initialized, Role: FOLLOWER
I20260811 06:28:11.170107 28767 raft_consensus.cc:374] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:11.170194 28767 raft_consensus.cc:729] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16846b4bb1b43d6886c2dec8ab19888, State: Initialized, Role: FOLLOWER
I20260811 06:28:11.170768 28765 consensus_queue.cc:260] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.171041 28767 consensus_queue.cc:260] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.171326 28768 consensus_queue.cc:260] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.172290 28745 heartbeater.cc:503] Master 127.7.4.190:41405 was elected leader, sending a full tablet report...
I20260811 06:28:11.173347 28767 ts_tablet_manager.cc:1428] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260811 06:28:11.174274 28597 heartbeater.cc:503] Master 127.7.4.190:41405 was elected leader, sending a full tablet report...
I20260811 06:28:11.174904 28768 ts_tablet_manager.cc:1428] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260811 06:28:11.177690 28765 ts_tablet_manager.cc:1428] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Time spent starting tablet: real 0.019s user 0.003s sys 0.011s
I20260811 06:28:11.264278 28773 raft_consensus.cc:482] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:11.264544 28773 raft_consensus.cc:504] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.266247 28773 leader_election.cc:288] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fd84ba9fe1c4405be60e845c5f88ae4 (127.7.4.129:34177), a16846b4bb1b43d6886c2dec8ab19888 (127.7.4.131:38625)
I20260811 06:28:11.275684 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.266808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:57587 (local address 127.7.4.131:38625)
0811 06:28:11.267021 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.267028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.267851 (+ 823us) server_negotiation.cc:407] Connection header received
0811 06:28:11.268030 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.268035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.268118 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.268223 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:11.269124 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.270241 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.273396 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.274598 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.274728 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.274823 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.275316 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.275435 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:28:11.275684 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.266801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:56289 (local address 127.7.4.129:34177)
0811 06:28:11.267055 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.267062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.267845 (+ 783us) server_negotiation.cc:407] Connection header received
0811 06:28:11.268030 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.268035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.268122 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.268222 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:11.269124 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.270234 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.273398 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.274596 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.274728 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.274823 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.275305 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.275435 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 06:28:11.275961 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.266673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:38625 (local address 127.7.4.130:57587)
0811 06:28:11.267734 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.267757 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.267881 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.268253 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.268259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.268302 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:11.268975 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.268985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.270383 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.270387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.273163 (+ 2776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.273179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.274815 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.274822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.275056 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.275162 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.275245 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.275437 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":594,"spinlock_wait_cycles":13568,"thread_start_us":382,"threads_started":1}
I20260811 06:28:11.275998 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.266662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:34177 (local address 127.7.4.130:56289)
0811 06:28:11.267695 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:11.267717 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:11.267887 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:11.268253 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:11.268259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:11.268304 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:11.268952 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.268971 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.270381 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.270385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.273164 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:11.273179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.274822 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.274828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:11.275061 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.275149 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 06:28:11.275245 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:28:11.275437 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":336,"spinlock_wait_cycles":47872,"thread_start_us":587,"threads_started":1}
I20260811 06:28:11.277024 28695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "415d95676fd240d085a120471b1efda7" candidate_uuid: "cb1e8e1cb495482e800419c90bf03137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16846b4bb1b43d6886c2dec8ab19888" is_pre_election: true
I20260811 06:28:11.277062 28381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "415d95676fd240d085a120471b1efda7" candidate_uuid: "cb1e8e1cb495482e800419c90bf03137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" is_pre_election: true
I20260811 06:28:11.277544 28695 raft_consensus.cc:2454] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb1e8e1cb495482e800419c90bf03137 in term 0.
I20260811 06:28:11.277590 28381 raft_consensus.cc:2454] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb1e8e1cb495482e800419c90bf03137 in term 0.
I20260811 06:28:11.278234 28462 leader_election.cc:302] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fd84ba9fe1c4405be60e845c5f88ae4, a16846b4bb1b43d6886c2dec8ab19888, cb1e8e1cb495482e800419c90bf03137; no voters:
I20260811 06:28:11.278645 28773 raft_consensus.cc:2781] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:11.278811 28773 raft_consensus.cc:482] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:11.278923 28773 raft_consensus.cc:3037] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:11.281227 28773 raft_consensus.cc:504] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.281818 28773 leader_election.cc:288] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [CANDIDATE]: Term 1 election: Requested vote from peers 3fd84ba9fe1c4405be60e845c5f88ae4 (127.7.4.129:34177), a16846b4bb1b43d6886c2dec8ab19888 (127.7.4.131:38625)
I20260811 06:28:11.282222 28381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "415d95676fd240d085a120471b1efda7" candidate_uuid: "cb1e8e1cb495482e800419c90bf03137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4"
I20260811 06:28:11.282305 28695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "415d95676fd240d085a120471b1efda7" candidate_uuid: "cb1e8e1cb495482e800419c90bf03137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16846b4bb1b43d6886c2dec8ab19888"
I20260811 06:28:11.282433 28381 raft_consensus.cc:3037] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:11.282485 28695 raft_consensus.cc:3037] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:11.285779 28381 raft_consensus.cc:2454] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb1e8e1cb495482e800419c90bf03137 in term 1.
I20260811 06:28:11.285769 28695 raft_consensus.cc:2454] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb1e8e1cb495482e800419c90bf03137 in term 1.
I20260811 06:28:11.286227 28462 leader_election.cc:302] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fd84ba9fe1c4405be60e845c5f88ae4, a16846b4bb1b43d6886c2dec8ab19888, cb1e8e1cb495482e800419c90bf03137; no voters:
I20260811 06:28:11.286545 28773 raft_consensus.cc:2781] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:11.286993 28773 raft_consensus.cc:686] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 1 LEADER]: Becoming Leader. State: Replica: cb1e8e1cb495482e800419c90bf03137, State: Running, Role: LEADER
I20260811 06:28:11.287504 28773 consensus_queue.cc:237] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } }
I20260811 06:28:11.291906 28218 catalog_manager.cc:5495] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb1e8e1cb495482e800419c90bf03137 (127.7.4.130). New cstate: current_term: 1 leader_uuid: "cb1e8e1cb495482e800419c90bf03137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb1e8e1cb495482e800419c90bf03137" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 } health_report { overall_health: UNKNOWN } } }
W20260811 06:28:11.303560 28748 tablet.cc:2369] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:11.320716 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.315133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.7.4.130:34381)
0811 06:28:11.315383 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.315392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.315489 (+ 97us) server_negotiation.cc:407] Connection header received
0811 06:28:11.315644 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.315648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.315735 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.315893 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:11.316297 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.316982 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.318410 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.318820 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.318957 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.319645 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:11.320086 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:11.320278 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.320403 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.320444 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:11.328362 28381 raft_consensus.cc:1261] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 1 FOLLOWER]: Refusing update from remote peer cb1e8e1cb495482e800419c90bf03137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:11.328366 28695 raft_consensus.cc:1261] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 1 FOLLOWER]: Refusing update from remote peer cb1e8e1cb495482e800419c90bf03137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:11.329347 28777 consensus_queue.cc:1035] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16846b4bb1b43d6886c2dec8ab19888" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 38625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:11.329669 28773 consensus_queue.cc:1035] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fd84ba9fe1c4405be60e845c5f88ae4" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:11.340250 28781 mvcc.cc:205] Tried to move back new op lower bound from 7317216015671451648 to 7317216015517814784. Current Snapshot: MvccSnapshot[applied={T|T < 7317216015671451648}]
I20260811 06:28:11.355976 28218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:11.356263 28218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:11.359933 28218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdc3651ec34841e79a106b672250b22d: Sending DeleteTablet for 3 replicas of tablet 415d95676fd240d085a120471b1efda7
I20260811 06:28:11.361054 28521 tablet_service.cc:1514] Processing DeleteTablet for tablet 415d95676fd240d085a120471b1efda7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:11 UTC) from {username='slave'} at 127.0.0.1:32894
I20260811 06:28:11.361042 28357 tablet_service.cc:1514] Processing DeleteTablet for tablet 415d95676fd240d085a120471b1efda7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:11 UTC) from {username='slave'} at 127.0.0.1:36222
I20260811 06:28:11.361104 28672 tablet_service.cc:1514] Processing DeleteTablet for tablet 415d95676fd240d085a120471b1efda7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:11 UTC) from {username='slave'} at 127.0.0.1:53032
I20260811 06:28:11.363193 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28285
I20260811 06:28:11.374683 28794 tablet_replica.cc:330] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: stopping tablet replica
I20260811 06:28:11.374958 28798 tablet_replica.cc:330] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: stopping tablet replica
I20260811 06:28:11.375524 28798 raft_consensus.cc:2229] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:11.376094 28798 raft_consensus.cc:2258] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:11.376045 28794 raft_consensus.cc:2229] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:11.376888 28794 raft_consensus.cc:2258] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:11.376952 28796 tablet_replica.cc:330] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: stopping tablet replica
I20260811 06:28:11.377416 28796 raft_consensus.cc:2229] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:11.378116 28796 raft_consensus.cc:2258] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:11.378701 28798 ts_tablet_manager.cc:1905] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:11.379256 28794 ts_tablet_manager.cc:1905] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:11.380465 28796 ts_tablet_manager.cc:1905] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:11.385715 28798 ts_tablet_manager.cc:1918] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:11.385883 28798 log.cc:1196] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/415d95676fd240d085a120471b1efda7
I20260811 06:28:11.385962 28794 ts_tablet_manager.cc:1918] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:11.386128 28794 log.cc:1196] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/415d95676fd240d085a120471b1efda7
I20260811 06:28:11.386425 28798 ts_tablet_manager.cc:1939] T 415d95676fd240d085a120471b1efda7 P a16846b4bb1b43d6886c2dec8ab19888: Deleting consensus metadata
I20260811 06:28:11.386662 28794 ts_tablet_manager.cc:1939] T 415d95676fd240d085a120471b1efda7 P 3fd84ba9fe1c4405be60e845c5f88ae4: Deleting consensus metadata
I20260811 06:28:11.388424 28205 catalog_manager.cc:4841] TS a16846b4bb1b43d6886c2dec8ab19888 (127.7.4.131:38625): tablet 415d95676fd240d085a120471b1efda7 (table test-table [id=97b8435c68f642f89857341ea38d136f]) successfully deleted
I20260811 06:28:11.388820 28203 catalog_manager.cc:4841] TS 3fd84ba9fe1c4405be60e845c5f88ae4 (127.7.4.129:34177): tablet 415d95676fd240d085a120471b1efda7 (table test-table [id=97b8435c68f642f89857341ea38d136f]) successfully deleted
I20260811 06:28:11.390678 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.364465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.7.4.129:34177)
0811 06:28:11.364652 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.364658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.369731 (+ 5073us) server_negotiation.cc:407] Connection header received
0811 06:28:11.369962 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.369968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.370087 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.370191 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:11.375188 (+ 4997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.376407 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.385961 (+ 9554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.386739 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.386851 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.389153 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:11.389170 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:11.389183 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:11.389794 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:11.389856 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:11.389875 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:11.389948 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.390188 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.390299 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:28:11.391470 28428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:11.393682 28796 ts_tablet_manager.cc:1918] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:11.393937 28796 log.cc:1196] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/415d95676fd240d085a120471b1efda7
I20260811 06:28:11.394610 28796 ts_tablet_manager.cc:1939] T 415d95676fd240d085a120471b1efda7 P cb1e8e1cb495482e800419c90bf03137: Deleting consensus metadata
I20260811 06:28:11.396682 28207 catalog_manager.cc:4841] TS cb1e8e1cb495482e800419c90bf03137 (127.7.4.130:34381): tablet 415d95676fd240d085a120471b1efda7 (table test-table [id=97b8435c68f642f89857341ea38d136f]) successfully deleted
I20260811 06:28:11.485383 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28285
I20260811 06:28:11.514544 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28440
I20260811 06:28:11.522797 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.515434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.7.4.130:34381)
0811 06:28:11.515612 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.515617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.515702 (+ 85us) server_negotiation.cc:407] Connection header received
0811 06:28:11.515888 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.515891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.515974 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.516057 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:11.516516 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.517319 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.519216 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.519883 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.519986 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.521187 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:11.521195 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:11.521209 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:11.521986 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:11.522043 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:11.522064 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:11.522144 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.522323 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.522416 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:11.523576 28591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:11.639870 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28440
I20260811 06:28:11.663250 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28601
I20260811 06:28:11.669620 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:11.664063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.7.4.131:38625)
0811 06:28:11.664233 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:11.664238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:11.664293 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:28:11.664441 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:11.664445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:11.664537 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:11.664621 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:11.665029 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.665758 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.666963 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:11.667405 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:11.667495 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:11.668461 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:11.668470 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:11.668484 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:11.669043 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:11.669095 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:11.669114 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:11.669197 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:11.669309 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 06:28:11.669391 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:28:11.670250 28738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:11.757709 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28601
I20260811 06:28:11.777842 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28187
I20260811 06:28:11.779314 28254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:11.897639 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28187
2026-08-11T06:28:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2834 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:12 dist-test-slave-07c2 krb5kdc[28807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:12 dist-test-slave-07c2 krb5kdc[28807](info): set up 2 sockets
Aug 11 06:28:12 dist-test-slave-07c2 krb5kdc[28807](info): commencing operation
krb5kdc: starting...
W20260811 06:28:13.094153 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:13 dist-test-slave-07c2 krb5kdc[28807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:13.394375 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34407
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:13.735191 28820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:13.735478 28820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:13.735567 28820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:13.742413 28820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:13.742509 28820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:13.742555 28820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:13.742619 28820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:13.742663 28820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:13.752630 28820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:13.753786 28820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:13.754990 28820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:13.766462 28825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:13.766422 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:13.766498 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:13.767570 28820 server_base.cc:1034] running on GCE node
I20260811 06:28:13.768306 28820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:13.769668 28820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:13.770910 28820 hybrid_clock.cc:648] HybridClock initialized: now 1786429693770862 us; error 57 us; skew 500 ppm
I20260811 06:28:13.771256 28820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:13 dist-test-slave-07c2 krb5kdc[28807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:13.774339 28820 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:13.775867 28820 webserver.cc:460] Webserver started at http://127.7.4.190:35037/ using document root <none> and password file <none>
I20260811 06:28:13.776599 28820 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:13.776751 28820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:13.777099 28820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:13.780333 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "54c0a2a861604b5d8a2698ae133f2482"
format_stamp: "Formatted at 2026-08-11 06:28:13 on dist-test-slave-07c2"
server_key: "5e7e577f6c66c919dad577fb2c32bc52"
server_key_iv: "6d13d1fa09a6463253e66ba090db8d51"
server_key_version: "encryptionkey@0"
I20260811 06:28:13.781157 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "54c0a2a861604b5d8a2698ae133f2482"
format_stamp: "Formatted at 2026-08-11 06:28:13 on dist-test-slave-07c2"
server_key: "5e7e577f6c66c919dad577fb2c32bc52"
server_key_iv: "6d13d1fa09a6463253e66ba090db8d51"
server_key_version: "encryptionkey@0"
I20260811 06:28:13.786873 28820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:28:13.790493 28835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:13.791889 28820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:13.792090 28820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "54c0a2a861604b5d8a2698ae133f2482"
format_stamp: "Formatted at 2026-08-11 06:28:13 on dist-test-slave-07c2"
server_key: "5e7e577f6c66c919dad577fb2c32bc52"
server_key_iv: "6d13d1fa09a6463253e66ba090db8d51"
server_key_version: "encryptionkey@0"
I20260811 06:28:13.792305 28820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:13.806406 28820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:13.810173 28820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:13.810474 28820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:13.829027 28820 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:34407
I20260811 06:28:13.829070 28890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:34407 every 8 connection(s)
I20260811 06:28:13.830344 28820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:13.833760 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28820
I20260811 06:28:13.833940 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:13.834106 28891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:13.834511 7186 external_mini_cluster.cc:1428] Setting key 74547d55464ce333f0ff5dd106189678
I20260811 06:28:13.848730 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: Bootstrap starting.
Aug 11 06:28:13 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:13.852389 28891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:13.853749 28891 log.cc:824] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:13.857264 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: No bootstrap required, opened a new log
I20260811 06:28:13.859194 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:13.836347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.7.4.190:34407)
0811 06:28:13.837157 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:13.837168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:13.837206 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:28:13.838835 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:13.838855 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:13.839199 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:13.839590 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:13.841401 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:13.842659 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:13.844583 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:13.845246 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:13.845381 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:13.850284 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:13.850300 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:13.850319 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:13.850367 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:13.854393 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:13.855370 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:13.855378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:13.855385 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:13.855578 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:13.855987 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:13.855994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:13.855998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:13.856742 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:13.857052 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:13.857444 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 06:28:13.857889 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":400,"threads_started":1}
I20260811 06:28:13.865864 28891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } }
I20260811 06:28:13.866175 28891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:13.866247 28891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c0a2a861604b5d8a2698ae133f2482, State: Initialized, Role: FOLLOWER
I20260811 06:28:13.866837 28891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } }
I20260811 06:28:13.867098 28891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:13.867247 28891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:13.867453 28891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:13.869496 28891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } }
I20260811 06:28:13.869921 28891 leader_election.cc:302] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c0a2a861604b5d8a2698ae133f2482; no voters:
I20260811 06:28:13.870457 28891 leader_election.cc:288] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:13.870596 28897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:13.871348 28897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [term 1 LEADER]: Becoming Leader. State: Replica: 54c0a2a861604b5d8a2698ae133f2482, State: Running, Role: LEADER
I20260811 06:28:13.871870 28897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } }
I20260811 06:28:13.872501 28891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:13.875147 28898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c0a2a861604b5d8a2698ae133f2482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } } }
I20260811 06:28:13.875121 28899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c0a2a861604b5d8a2698ae133f2482. Latest consensus state: current_term: 1 leader_uuid: "54c0a2a861604b5d8a2698ae133f2482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c0a2a861604b5d8a2698ae133f2482" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34407 } } }
I20260811 06:28:13.875597 28898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:13.875597 28899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:13.880298 28903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:13.885480 28903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:13.893669 28903 catalog_manager.cc:1347] Generated new cluster ID: a8e8d6acc5f1454b821c2daa0920b080
I20260811 06:28:13.893800 28903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:13.916913 28903 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:13.918629 28903 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:13.935796 28903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: Generated new TSK 0
I20260811 06:28:13.936793 28903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.4.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
Entry for principal kudu/127.7.4.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab.
I20260811 06:28:14.022630 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:14.360751 28921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:14.361078 28921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:14.361245 28921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:14.368378 28921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:14.368517 28921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:14.368698 28921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:28:14.378408 28921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.129.keytab
--principal=kudu/127.7.4.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:14.379655 28921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:14.381166 28921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:14.394048 28932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:14.394107 28926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:14.394132 28927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:14.395181 28921 server_base.cc:1034] running on GCE node
I20260811 06:28:14.395845 28921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:14.396803 28921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:14.398025 28921 hybrid_clock.cc:648] HybridClock initialized: now 1786429694397938 us; error 72 us; skew 500 ppm
I20260811 06:28:14.398339 28921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:14 dist-test-slave-07c2 krb5kdc[28807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429694, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:14.401510 28921 init.cc:377] Logged in from keytab as kudu/127.7.4.129@KRBTEST.COM (short username kudu)
I20260811 06:28:14.402892 28921 webserver.cc:460] Webserver started at http://127.7.4.129:34497/ using document root <none> and password file <none>
I20260811 06:28:14.403610 28921 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:14.403751 28921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:14.404057 28921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:14.407109 28921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "a6522f07a4454d85baa2c03e424127bc"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "157573f25c8a96d4f92bfe24ee45850c"
server_key_iv: "6db01067436fab040fd9e7e31acb2f43"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.408011 28921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "a6522f07a4454d85baa2c03e424127bc"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "157573f25c8a96d4f92bfe24ee45850c"
server_key_iv: "6db01067436fab040fd9e7e31acb2f43"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.413067 28921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 06:28:14.416414 28940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:14.417646 28921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:28:14.417826 28921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "a6522f07a4454d85baa2c03e424127bc"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "157573f25c8a96d4f92bfe24ee45850c"
server_key_iv: "6db01067436fab040fd9e7e31acb2f43"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.418030 28921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:14.445528 28921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:14.449316 28921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:14.449666 28921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:14.450588 28921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:14.451850 28921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:14.451923 28921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:14.452095 28921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:14.452172 28921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:14.485836 28921 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:35075
I20260811 06:28:14.485920 29068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:35075 every 8 connection(s)
I20260811 06:28:14.487088 28921 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:28:14.492609 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 28921
I20260811 06:28:14.492785 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:28:14.493263 7186 external_mini_cluster.cc:1428] Setting key 3f5f59d876a0bcfed301d40ec46faf26
Aug 11 06:28:14 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429694, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.129@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:14.506883 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:14.489660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:46459 (local address 127.7.4.190:34407)
0811 06:28:14.490176 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:14.490180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:14.491033 (+ 853us) server_negotiation.cc:407] Connection header received
0811 06:28:14.492998 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:14.493005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:14.493095 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:14.493219 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:14.494814 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:14.495706 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:14.497387 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:14.497782 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:14.497939 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:14.501373 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:14.501390 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:14.501393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:14.501423 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:14.503954 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:14.504648 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:14.504654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:14.504658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:14.504786 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:14.505194 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:14.505199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:14.505202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:14.505450 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:14.505597 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:14.506268 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 06:28:14.506488 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":269,"threads_started":1}
I20260811 06:28:14.507812 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:14.490142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34407 (local address 127.7.4.129:46459)
0811 06:28:14.490856 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:14.490897 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:14.492681 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:14.493474 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:14.493483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:14.494006 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:14.494623 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:14.494640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:14.495876 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:14.495880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:14.497228 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:14.497246 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:14.498009 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:14.498018 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:14.498278 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:14.498948 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:14.498969 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:14.501157 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:14.504093 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:14.504101 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:14.504114 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:14.504514 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:14.504904 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:14.504908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:14.504911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:14.505085 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:14.505572 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:14.505579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:14.505912 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 06:28:14.506162 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:28:14.506645 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":310,"threads_started":1}
I20260811 06:28:14.509277 29070 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34407
I20260811 06:28:14.509577 29070 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:14.510258 29070 heartbeater.cc:511] Master 127.7.4.190:34407 requested a full tablet report, sending...
I20260811 06:28:14.512221 28853 ts_manager.cc:194] Registered new tserver with Master: a6522f07a4454d85baa2c03e424127bc (127.7.4.129:35075)
I20260811 06:28:14.513702 28853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.129@KRBTEST.COM'} at 127.7.4.129:46459
WARNING: no policy specified for kudu/127.7.4.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
Entry for principal kudu/127.7.4.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab.
I20260811 06:28:14.566751 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:14.901768 29077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:14.902006 29077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:14.902160 29077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:14.909045 29077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:14.909166 29077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:14.909353 29077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:28:14.918306 29077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.130.keytab
--principal=kudu/127.7.4.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:14.919420 29077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:14.920627 29077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:14.932989 29083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:14.933079 29084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:14.933036 29087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:14.934063 29077 server_base.cc:1034] running on GCE node
I20260811 06:28:14.934612 29077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:14.935595 29077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:14.936843 29077 hybrid_clock.cc:648] HybridClock initialized: now 1786429694936762 us; error 54 us; skew 500 ppm
I20260811 06:28:14.937142 29077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:14 dist-test-slave-07c2 krb5kdc[28807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429694, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:14.940326 29077 init.cc:377] Logged in from keytab as kudu/127.7.4.130@KRBTEST.COM (short username kudu)
I20260811 06:28:14.941629 29077 webserver.cc:460] Webserver started at http://127.7.4.130:40925/ using document root <none> and password file <none>
I20260811 06:28:14.942219 29077 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:14.942330 29077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:14.942601 29077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:14.945396 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "72e14d9ce1b9452aae262c8285d63217"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "c597bb6d5552c769e60eb4dcbc1576e0"
server_key_iv: "1159585a0b18b8315818c9a8ba7f2263"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.946100 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "72e14d9ce1b9452aae262c8285d63217"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "c597bb6d5552c769e60eb4dcbc1576e0"
server_key_iv: "1159585a0b18b8315818c9a8ba7f2263"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.950985 29077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 06:28:14.954210 29094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:14.955519 29077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:28:14.955644 29077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "72e14d9ce1b9452aae262c8285d63217"
format_stamp: "Formatted at 2026-08-11 06:28:14 on dist-test-slave-07c2"
server_key: "c597bb6d5552c769e60eb4dcbc1576e0"
server_key_iv: "1159585a0b18b8315818c9a8ba7f2263"
server_key_version: "encryptionkey@0"
I20260811 06:28:14.955835 29077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:14.972491 29077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:14.976343 29077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:14.976622 29077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:14.977591 29077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:14.978896 29077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:14.979000 29077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:14.979164 29077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:14.979239 29077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:15.013350 29077 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:39691
I20260811 06:28:15.013394 29215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:39691 every 8 connection(s)
I20260811 06:28:15.014659 29077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:28:15.016921 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29077
I20260811 06:28:15.017117 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:28:15.017674 7186 external_mini_cluster.cc:1428] Setting key efbd91477f78ed43cc249ef6963f5cca
Aug 11 06:28:15 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429694, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.130@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:15.038499 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.017217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:45387 (local address 127.7.4.190:34407)
0811 06:28:15.017759 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.017766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.018873 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 06:28:15.021431 (+ 2558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.021434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.021495 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.021579 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:15.023617 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.024382 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.026770 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.027176 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.027263 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.031563 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:15.031572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:15.031579 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:15.031623 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:15.035177 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.036175 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.036179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.036181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.036313 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.036810 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.036813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.036815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.037069 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:15.037178 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.038029 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.038191 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":257,"threads_started":1}
I20260811 06:28:15.040133 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.017853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34407 (local address 127.7.4.130:45387)
0811 06:28:15.018694 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.018755 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.021132 (+ 2377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.021841 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.021856 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.022639 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:15.023433 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.023460 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.024549 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.024557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.026630 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.026647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.027439 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.027453 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.027780 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.028495 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:15.028547 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:15.031325 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:15.035379 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:15.035389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:15.035407 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:15.036037 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:15.036426 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:15.036434 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:15.036439 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:15.036697 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:15.037223 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:15.037233 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:15.037659 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.037956 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.038516 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":342,"threads_started":1}
I20260811 06:28:15.041601 29216 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34407
I20260811 06:28:15.041913 29216 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:15.042677 29216 heartbeater.cc:511] Master 127.7.4.190:34407 requested a full tablet report, sending...
I20260811 06:28:15.044194 28853 ts_manager.cc:194] Registered new tserver with Master: 72e14d9ce1b9452aae262c8285d63217 (127.7.4.130:39691)
I20260811 06:28:15.045008 28853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.130@KRBTEST.COM'} at 127.7.4.130:45387
WARNING: no policy specified for kudu/127.7.4.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
Entry for principal kudu/127.7.4.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab.
I20260811 06:28:15.090109 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:15.422097 29222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:15.422401 29222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:15.422564 29222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:15.429438 29222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:15.429561 29222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:15.429754 29222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:15.438678 29222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.131.keytab
--principal=kudu/127.7.4.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:15.439842 29222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:15.441051 29222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:15.454007 29228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:15.454005 29231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:15.454605 29227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:15.455067 29222 server_base.cc:1034] running on GCE node
I20260811 06:28:15.455713 29222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:15.456626 29222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:15.457852 29222 hybrid_clock.cc:648] HybridClock initialized: now 1786429695457805 us; error 34 us; skew 500 ppm
I20260811 06:28:15.458158 29222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:15 dist-test-slave-07c2 krb5kdc[28807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429695, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:15.461287 29222 init.cc:377] Logged in from keytab as kudu/127.7.4.131@KRBTEST.COM (short username kudu)
I20260811 06:28:15.462561 29222 webserver.cc:460] Webserver started at http://127.7.4.131:39365/ using document root <none> and password file <none>
I20260811 06:28:15.463232 29222 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:15.463398 29222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:15.463728 29222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:15.466544 29222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "f0f5bf3c07b14b2a8eb46351972a999a"
format_stamp: "Formatted at 2026-08-11 06:28:15 on dist-test-slave-07c2"
server_key: "8717a9271dcb522ad8aac6828cacc98c"
server_key_iv: "35e2d127ac73744b5e0184590ab8577d"
server_key_version: "encryptionkey@0"
I20260811 06:28:15.467368 29222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "f0f5bf3c07b14b2a8eb46351972a999a"
format_stamp: "Formatted at 2026-08-11 06:28:15 on dist-test-slave-07c2"
server_key: "8717a9271dcb522ad8aac6828cacc98c"
server_key_iv: "35e2d127ac73744b5e0184590ab8577d"
server_key_version: "encryptionkey@0"
I20260811 06:28:15.472191 29222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:28:15.475755 29238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:15.476962 29222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:28:15.477133 29222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "f0f5bf3c07b14b2a8eb46351972a999a"
format_stamp: "Formatted at 2026-08-11 06:28:15 on dist-test-slave-07c2"
server_key: "8717a9271dcb522ad8aac6828cacc98c"
server_key_iv: "35e2d127ac73744b5e0184590ab8577d"
server_key_version: "encryptionkey@0"
I20260811 06:28:15.477334 29222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:15.500366 29222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:15.504026 29222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:15.504314 29222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:15.505246 29222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:15.506464 29222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:15.506564 29222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:15.506705 29222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:15.506784 29222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:15.516484 29070 heartbeater.cc:503] Master 127.7.4.190:34407 was elected leader, sending a full tablet report...
I20260811 06:28:15.539819 29222 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:39165
I20260811 06:28:15.539865 29367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:39165 every 8 connection(s)
I20260811 06:28:15.541064 29222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:15.548815 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29222
I20260811 06:28:15.549000 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:15.549465 7186 external_mini_cluster.cc:1428] Setting key ad3d830d37e17800f280eca8a686e3a6
Aug 11 06:28:15 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429695, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.131@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:15.561482 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.543741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:53793 (local address 127.7.4.190:34407)
0811 06:28:15.544222 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.544227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.545089 (+ 862us) server_negotiation.cc:407] Connection header received
0811 06:28:15.546963 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.546968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.547062 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.547194 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:15.548745 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.549708 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.551696 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.552317 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.552457 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.555863 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:15.555867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:15.555870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:15.555904 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:15.558553 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.559399 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.559402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.559405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.559525 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.559977 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.559980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.559982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.560282 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:15.560406 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.561037 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.561213 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":255,"threads_started":1}
I20260811 06:28:15.562402 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.544225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34407 (local address 127.7.4.131:53793)
0811 06:28:15.544928 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.544965 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.546681 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.547465 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.547474 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.547980 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:15.548568 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.548587 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.549889 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.549893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.551543 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.551561 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.552548 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.552558 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.552804 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.553408 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:15.553429 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:15.555665 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:15.558685 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:15.558691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:15.558702 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:15.559234 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:15.559630 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:15.559633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:15.559636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:15.559824 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:15.560405 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:15.560411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:15.560702 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.560933 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.561300 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":298,"threads_started":1}
I20260811 06:28:15.564047 29368 heartbeater.cc:347] Connected to a master server at 127.7.4.190:34407
I20260811 06:28:15.564436 29368 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:15.565168 29368 heartbeater.cc:511] Master 127.7.4.190:34407 requested a full tablet report, sending...
I20260811 06:28:15.566571 28852 ts_manager.cc:194] Registered new tserver with Master: f0f5bf3c07b14b2a8eb46351972a999a (127.7.4.131:39165)
I20260811 06:28:15.567502 28852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.4.131@KRBTEST.COM'} at 127.7.4.131:53793
I20260811 06:28:15.576799 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:15.598532 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.584840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44074 (local address 127.7.4.190:34407)
0811 06:28:15.585020 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.585026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.585600 (+ 574us) server_negotiation.cc:407] Connection header received
0811 06:28:15.585843 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.585847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.585909 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.586031 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:15.588124 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.589096 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.590576 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.590984 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.591075 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.593364 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:15.593368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:15.593371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:15.593402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:15.596091 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.596719 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.596723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.596725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.596837 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.597286 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.597290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.597292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.597599 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:15.597788 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.598051 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.598212 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:28:15.602198 28852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:15.609123 28852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:15.647084 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.621047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:39691 (local address 127.0.0.1:53178)
0811 06:28:15.621750 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.621868 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.628319 (+ 6451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.637587 (+ 9268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.637598 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.637641 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:15.638148 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.638165 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.640797 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.640805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.643740 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.643759 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.646086 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.646096 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.646324 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.646378 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.646491 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.646698 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
I20260811 06:28:15.648841 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.621297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.7.4.130:39691)
0811 06:28:15.636591 (+ 15294us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.636604 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.636642 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:28:15.636764 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.636776 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.637078 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.638403 (+ 1325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:15.638661 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.640615 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.644020 (+ 3405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.645821 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.648017 (+ 2196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.648235 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.648313 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.648462 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15081,"thread_start_us":15946,"threads_started":1}
I20260811 06:28:15.655392 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.621637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:35075 (local address 127.0.0.1:56656)
0811 06:28:15.646422 (+ 24785us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.646441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.646562 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.647515 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.647519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.647570 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:15.648863 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.648879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.650271 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.650279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.652630 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.652644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.654617 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.654627 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.654804 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.654845 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.654913 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.655029 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24662,"thread_start_us":28624,"threads_started":1}
I20260811 06:28:15.655391 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f (DEFAULT_TABLE table=test-table [id=8cdb58e95fec4e148ba11f64c924d540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:15.655997 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.622389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.7.4.129:35075)
0811 06:28:15.645853 (+ 23464us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.645863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.646684 (+ 821us) server_negotiation.cc:407] Connection header received
0811 06:28:15.646786 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.646796 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.647047 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.647241 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:15.649079 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.650141 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.652804 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.654308 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.654440 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.655388 (+ 948us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.655484 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.655672 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23279,"thread_start_us":23453,"threads_started":1}
I20260811 06:28:15.656783 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d30fffbed6c4582a3fcf69083a9527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:15.658941 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.622181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:39165 (local address 127.0.0.1:40996)
0811 06:28:15.647726 (+ 25545us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.647747 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.647846 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.651885 (+ 4039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.651890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.651915 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:15.652128 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.652143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.653824 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.653830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.655722 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.655739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.658219 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.658226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.658407 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.658440 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.658514 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.658695 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25057,"thread_start_us":28432,"threads_started":1}
I20260811 06:28:15.659433 28997 tablet_service.cc:1467] Processing CreateTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f (DEFAULT_TABLE table=test-table [id=8cdb58e95fec4e148ba11f64c924d540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:15.660115 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.622583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.7.4.131:39165)
0811 06:28:15.651314 (+ 28731us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.651322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.651355 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:28:15.651450 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.651457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.651668 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.652298 (+ 630us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:15.652582 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.653675 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.655850 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.658010 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.659311 (+ 1301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.659550 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.659618 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.659776 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28529,"thread_start_us":28324,"threads_started":1}
I20260811 06:28:15.660740 28997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d30fffbed6c4582a3fcf69083a9527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:15.665653 29293 tablet_service.cc:1467] Processing CreateTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f (DEFAULT_TABLE table=test-table [id=8cdb58e95fec4e148ba11f64c924d540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:15.667073 29293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d30fffbed6c4582a3fcf69083a9527f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:15.670830 29390 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Bootstrap starting.
I20260811 06:28:15.671757 29391 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Bootstrap starting.
I20260811 06:28:15.674530 29390 tablet_bootstrap.cc:654] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:15.675249 29391 tablet_bootstrap.cc:654] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:15.675617 29390 log.cc:824] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:15.676462 29391 log.cc:824] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:15.681494 29391 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: No bootstrap required, opened a new log
I20260811 06:28:15.681648 29390 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: No bootstrap required, opened a new log
I20260811 06:28:15.681797 29391 ts_tablet_manager.cc:1397] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:28:15.681957 29390 ts_tablet_manager.cc:1397] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.005s
I20260811 06:28:15.684476 29394 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Bootstrap starting.
I20260811 06:28:15.687889 29394 tablet_bootstrap.cc:654] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:15.689141 29394 log.cc:824] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:15.690642 29390 raft_consensus.cc:348] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.691023 29390 raft_consensus.cc:374] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:15.691166 29390 raft_consensus.cc:729] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e14d9ce1b9452aae262c8285d63217, State: Initialized, Role: FOLLOWER
I20260811 06:28:15.691497 29391 raft_consensus.cc:348] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.691928 29391 raft_consensus.cc:374] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:15.691772 29390 consensus_queue.cc:260] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.692039 29391 raft_consensus.cc:729] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6522f07a4454d85baa2c03e424127bc, State: Initialized, Role: FOLLOWER
I20260811 06:28:15.692420 29394 tablet_bootstrap.cc:492] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: No bootstrap required, opened a new log
I20260811 06:28:15.692696 29391 consensus_queue.cc:260] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.692982 29216 heartbeater.cc:503] Master 127.7.4.190:34407 was elected leader, sending a full tablet report...
I20260811 06:28:15.692726 29394 ts_tablet_manager.cc:1397] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 06:28:15.694348 29391 ts_tablet_manager.cc:1428] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260811 06:28:15.694665 29390 ts_tablet_manager.cc:1428] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260811 06:28:15.702560 29394 raft_consensus.cc:348] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.702878 29394 raft_consensus.cc:374] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:15.702994 29394 raft_consensus.cc:729] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f5bf3c07b14b2a8eb46351972a999a, State: Initialized, Role: FOLLOWER
I20260811 06:28:15.703548 29394 consensus_queue.cc:260] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.704486 29368 heartbeater.cc:503] Master 127.7.4.190:34407 was elected leader, sending a full tablet report...
I20260811 06:28:15.705030 29394 ts_tablet_manager.cc:1428] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 06:28:15.727857 29398 raft_consensus.cc:482] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:15.728140 29398 raft_consensus.cc:504] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.729846 29398 leader_election.cc:288] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72e14d9ce1b9452aae262c8285d63217 (127.7.4.130:39691), a6522f07a4454d85baa2c03e424127bc (127.7.4.129:35075)
I20260811 06:28:15.736150 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.730392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:43495 (local address 127.7.4.129:35075)
0811 06:28:15.730612 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.730620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.730653 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:28:15.730731 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.730735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.730822 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.730928 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:15.731403 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.732412 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.734177 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.735253 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.735434 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.735507 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.735819 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.735906 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:28:15.736253 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.730103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:35075 (local address 127.7.4.131:43495)
0811 06:28:15.730382 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.730410 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.730547 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.730956 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.730960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.730990 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:15.731235 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.731247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.732541 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.732547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.734019 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.734032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.735461 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.735467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.735650 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.735685 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.735790 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.735919 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":1}
I20260811 06:28:15.736953 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.730154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:39691 (local address 127.7.4.131:39139)
0811 06:28:15.730971 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:15.730990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:15.731117 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:15.731522 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:15.731527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:15.731555 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:15.731947 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.731958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.733161 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.733165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.734729 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:15.734744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.736157 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.736166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:15.736354 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.736396 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 06:28:15.736554 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:28:15.736676 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":446,"threads_started":1}
I20260811 06:28:15.736966 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.730105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:39139 (local address 127.7.4.130:39691)
0811 06:28:15.730351 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.730357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.731085 (+ 728us) server_negotiation.cc:407] Connection header received
0811 06:28:15.731258 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.731264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.731392 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.731471 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:15.732098 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.732996 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.734896 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.735926 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.736052 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.736142 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.736591 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.736687 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 06:28:15.737316 29020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d30fffbed6c4582a3fcf69083a9527f" candidate_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6522f07a4454d85baa2c03e424127bc" is_pre_election: true
I20260811 06:28:15.737941 29020 raft_consensus.cc:2454] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0f5bf3c07b14b2a8eb46351972a999a in term 0.
I20260811 06:28:15.738027 29167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d30fffbed6c4582a3fcf69083a9527f" candidate_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e14d9ce1b9452aae262c8285d63217" is_pre_election: true
I20260811 06:28:15.738579 29167 raft_consensus.cc:2454] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0f5bf3c07b14b2a8eb46351972a999a in term 0.
I20260811 06:28:15.738654 29242 leader_election.cc:302] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6522f07a4454d85baa2c03e424127bc, f0f5bf3c07b14b2a8eb46351972a999a; no voters:
I20260811 06:28:15.739081 29398 raft_consensus.cc:2781] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:15.739271 29398 raft_consensus.cc:482] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:15.739430 29398 raft_consensus.cc:3037] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 0 FOLLOWER]: Advancing to term 1
W20260811 06:28:15.739571 29071 tablet.cc:2369] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:15.741882 29398 raft_consensus.cc:504] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.742472 29398 leader_election.cc:288] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [CANDIDATE]: Term 1 election: Requested vote from peers 72e14d9ce1b9452aae262c8285d63217 (127.7.4.130:39691), a6522f07a4454d85baa2c03e424127bc (127.7.4.129:35075)
I20260811 06:28:15.742897 29167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d30fffbed6c4582a3fcf69083a9527f" candidate_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e14d9ce1b9452aae262c8285d63217"
I20260811 06:28:15.742905 29020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d30fffbed6c4582a3fcf69083a9527f" candidate_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6522f07a4454d85baa2c03e424127bc"
I20260811 06:28:15.743093 29020 raft_consensus.cc:3037] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:15.743089 29167 raft_consensus.cc:3037] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:15.745347 29020 raft_consensus.cc:2454] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0f5bf3c07b14b2a8eb46351972a999a in term 1.
I20260811 06:28:15.745630 29167 raft_consensus.cc:2454] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0f5bf3c07b14b2a8eb46351972a999a in term 1.
I20260811 06:28:15.745827 29242 leader_election.cc:302] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6522f07a4454d85baa2c03e424127bc, f0f5bf3c07b14b2a8eb46351972a999a; no voters:
I20260811 06:28:15.746137 29398 raft_consensus.cc:2781] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:15.746591 29398 raft_consensus.cc:686] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 1 LEADER]: Becoming Leader. State: Replica: f0f5bf3c07b14b2a8eb46351972a999a, State: Running, Role: LEADER
I20260811 06:28:15.747044 29398 consensus_queue.cc:237] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } }
I20260811 06:28:15.751781 28853 catalog_manager.cc:5495] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a reported cstate change: term changed from 0 to 1, leader changed from <none> to f0f5bf3c07b14b2a8eb46351972a999a (127.7.4.131). New cstate: current_term: 1 leader_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0f5bf3c07b14b2a8eb46351972a999a" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 39165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 } health_report { overall_health: UNKNOWN } } }
W20260811 06:28:15.766731 29217 tablet.cc:2369] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:15.783215 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.777794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.7.4.131:39165)
0811 06:28:15.777962 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.777967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.778140 (+ 173us) server_negotiation.cc:407] Connection header received
0811 06:28:15.778300 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.778303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.778383 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.778557 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:15.778866 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.779698 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.781140 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.781594 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.781701 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.782194 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:15.782660 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:15.782767 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.782913 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.782976 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:28:15.790625 29167 raft_consensus.cc:1261] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 1 FOLLOWER]: Refusing update from remote peer f0f5bf3c07b14b2a8eb46351972a999a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:15.790766 29020 raft_consensus.cc:1261] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 1 FOLLOWER]: Refusing update from remote peer f0f5bf3c07b14b2a8eb46351972a999a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:15.791654 29398 consensus_queue.cc:1035] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [LEADER]: Connected to new peer: Peer: permanent_uuid: "72e14d9ce1b9452aae262c8285d63217" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:15.791939 29400 consensus_queue.cc:1035] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6522f07a4454d85baa2c03e424127bc" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:28:15.792670 29370 tablet.cc:2369] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:15.812749 29404 mvcc.cc:205] Tried to move back new op lower bound from 7317216033949216768 to 7317216033784414208. Current Snapshot: MvccSnapshot[applied={T|T < 7317216033949216768}]
I20260811 06:28:15.818965 28853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:15.819247 28853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:15.822919 28853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54c0a2a861604b5d8a2698ae133f2482: Sending DeleteTablet for 3 replicas of tablet 3d30fffbed6c4582a3fcf69083a9527f
I20260811 06:28:15.824079 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:53178
I20260811 06:28:15.824436 28997 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:56656
I20260811 06:28:15.824106 29293 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d30fffbed6c4582a3fcf69083a9527f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:15 UTC) from {username='kudu', principal='kudu/127.7.4.190@KRBTEST.COM'} at 127.0.0.1:40996
I20260811 06:28:15.825037 29415 tablet_replica.cc:330] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: stopping tablet replica
I20260811 06:28:15.825315 29416 tablet_replica.cc:330] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: stopping tablet replica
I20260811 06:28:15.825428 29415 raft_consensus.cc:2229] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:15.825728 29416 raft_consensus.cc:2229] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:15.826009 29415 raft_consensus.cc:2258] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:15.826189 29416 raft_consensus.cc:2258] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:15.828451 29415 ts_tablet_manager.cc:1905] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:15.829880 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28921
I20260811 06:28:15.836124 29415 ts_tablet_manager.cc:1918] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:15.836344 29415 log.cc:1196] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/3d30fffbed6c4582a3fcf69083a9527f
I20260811 06:28:15.837051 29415 ts_tablet_manager.cc:1939] T 3d30fffbed6c4582a3fcf69083a9527f P 72e14d9ce1b9452aae262c8285d63217: Deleting consensus metadata
I20260811 06:28:15.839339 28838 catalog_manager.cc:4841] TS 72e14d9ce1b9452aae262c8285d63217 (127.7.4.130:39691): tablet 3d30fffbed6c4582a3fcf69083a9527f (table test-table [id=8cdb58e95fec4e148ba11f64c924d540]) successfully deleted
I20260811 06:28:15.839401 29416 ts_tablet_manager.cc:1905] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:15.844230 29417 tablet_replica.cc:330] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: stopping tablet replica
I20260811 06:28:15.844717 29417 raft_consensus.cc:2229] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:15.847139 29416 ts_tablet_manager.cc:1918] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:15.847344 29416 log.cc:1196] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/3d30fffbed6c4582a3fcf69083a9527f
I20260811 06:28:15.847961 29416 ts_tablet_manager.cc:1939] T 3d30fffbed6c4582a3fcf69083a9527f P a6522f07a4454d85baa2c03e424127bc: Deleting consensus metadata
I20260811 06:28:15.849933 28840 catalog_manager.cc:4841] TS a6522f07a4454d85baa2c03e424127bc (127.7.4.129:35075): tablet 3d30fffbed6c4582a3fcf69083a9527f (table test-table [id=8cdb58e95fec4e148ba11f64c924d540]) successfully deleted
I20260811 06:28:15.850468 29417 raft_consensus.cc:2258] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 06:28:15 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.129@KRBTEST.COM
I20260811 06:28:15.857504 29417 ts_tablet_manager.cc:1905] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:15.862855 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:15.830802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.7.4.129:35075)
0811 06:28:15.830953 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:15.830960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:15.841957 (+ 10997us) server_negotiation.cc:407] Connection header received
0811 06:28:15.842385 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:15.842391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:15.842492 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:15.842565 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:15.844612 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.845792 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.847932 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:15.848633 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:15.848733 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:15.854155 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:15.854167 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:15.854181 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:15.854214 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:15.858430 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.859395 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.859405 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.859411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.859594 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:15.860098 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:15.860103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:15.860106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:15.860567 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:15.860865 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:15.862182 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0811 06:28:15.862424 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:28:15.863914 29064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:15.867717 29417 ts_tablet_manager.cc:1918] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:15.867964 29417 log.cc:1196] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/3d30fffbed6c4582a3fcf69083a9527f
I20260811 06:28:15.868502 29417 ts_tablet_manager.cc:1939] T 3d30fffbed6c4582a3fcf69083a9527f P f0f5bf3c07b14b2a8eb46351972a999a: Deleting consensus metadata
I20260811 06:28:15.870059 28838 catalog_manager.cc:4841] TS f0f5bf3c07b14b2a8eb46351972a999a (127.7.4.131:39165): tablet 3d30fffbed6c4582a3fcf69083a9527f (table test-table [id=8cdb58e95fec4e148ba11f64c924d540]) successfully deleted
I20260811 06:28:15.971052 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28921
I20260811 06:28:16.000837 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29077
Aug 11 06:28:16 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.130@KRBTEST.COM
I20260811 06:28:16.017390 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:16.001772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.7.4.130:39691)
0811 06:28:16.001972 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:16.001979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:16.002003 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:28:16.002077 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:16.002083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:16.002173 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:16.002285 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:16.003663 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.004446 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.005717 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.006109 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.006255 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:16.010576 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:16.010592 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:16.010607 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:16.010638 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:16.014232 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:16.015058 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:16.015067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:16.015071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:16.015213 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:16.015684 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:16.015689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:16.015693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:16.016141 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:16.016427 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:16.016769 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 06:28:16.017016 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:28:16.018175 29211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:16.107025 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29077
I20260811 06:28:16.125963 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29222
Aug 11 06:28:16 dist-test-slave-07c2 krb5kdc[28807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.131@KRBTEST.COM
I20260811 06:28:16.144760 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:16.126809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.7.4.131:39165)
0811 06:28:16.126993 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:16.126997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:16.127132 (+ 135us) server_negotiation.cc:407] Connection header received
0811 06:28:16.127380 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:16.127384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:16.127451 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:16.127541 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:16.129467 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.130284 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.132321 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.132776 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.132881 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:16.138587 (+ 5706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:16.138598 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:16.138617 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:16.138668 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:16.141719 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:16.142434 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:16.142442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:16.142444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:16.142581 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:16.143002 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:16.143007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:16.143010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:16.143521 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:16.143912 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:16.144159 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 06:28:16.144414 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:28:16.145517 29362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:16.267576 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29222
I20260811 06:28:16.291364 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 28820
I20260811 06:28:16.292523 28885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:16.404809 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 28820
2026-08-11T06:28:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4508 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T06:28:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:16Z Disabled control of system clock
I20260811 06:28:16.500308 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46349
--webserver_interface=127.7.4.190
--webserver_port=0
--builtin_ntp_servers=127.7.4.148:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:16.832722 29426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:16.832975 29426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:16.833071 29426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:16.840124 29426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:16.840214 29426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:16.840250 29426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:16.840305 29426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:16.840348 29426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:16.849189 29426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:16.850293 29426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:16.852047 29426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:16.863626 29433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:16.863708 29435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:16.863683 29432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:16.864362 29426 server_base.cc:1034] running on GCE node
I20260811 06:28:16.865046 29426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:16.866279 29426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:16.867470 29426 hybrid_clock.cc:648] HybridClock initialized: now 1786429696867433 us; error 30 us; skew 500 ppm
I20260811 06:28:16.867815 29426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:16.869912 29426 webserver.cc:460] Webserver started at http://127.7.4.190:34557/ using document root <none> and password file <none>
I20260811 06:28:16.870524 29426 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:16.870637 29426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:16.871009 29426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:16.873813 29426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "013894e21f574b3c8ae103533c88e4db"
format_stamp: "Formatted at 2026-08-11 06:28:16 on dist-test-slave-07c2"
server_key: "ec2b13d94502e5b862ab065aead118ae"
server_key_iv: "b9d07fe2be2c8425980a750f6b872453"
server_key_version: "encryptionkey@0"
I20260811 06:28:16.874617 29426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "013894e21f574b3c8ae103533c88e4db"
format_stamp: "Formatted at 2026-08-11 06:28:16 on dist-test-slave-07c2"
server_key: "ec2b13d94502e5b862ab065aead118ae"
server_key_iv: "b9d07fe2be2c8425980a750f6b872453"
server_key_version: "encryptionkey@0"
I20260811 06:28:16.879417 29426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 06:28:16.882565 29441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:16.883883 29426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:16.884061 29426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "013894e21f574b3c8ae103533c88e4db"
format_stamp: "Formatted at 2026-08-11 06:28:16 on dist-test-slave-07c2"
server_key: "ec2b13d94502e5b862ab065aead118ae"
server_key_iv: "b9d07fe2be2c8425980a750f6b872453"
server_key_version: "encryptionkey@0"
I20260811 06:28:16.884258 29426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:16.899840 29426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:16.900519 29426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:16.900772 29426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:16.919355 29426 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:46349
I20260811 06:28:16.919479 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:46349 every 8 connection(s)
I20260811 06:28:16.920743 29426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:16.924242 29501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:16.925384 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29426
I20260811 06:28:16.925585 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:16.926034 7186 external_mini_cluster.cc:1428] Setting key c60139f36f28cf9248812c70c0fb3284
I20260811 06:28:16.938781 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: Bootstrap starting.
I20260811 06:28:16.940937 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:16.927876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.7.4.190:46349)
0811 06:28:16.929052 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:16.929066 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:16.929122 (+ 56us) server_negotiation.cc:407] Connection header received
0811 06:28:16.930818 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:16.930844 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:16.931426 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:16.931875 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:16.932399 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.934300 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.935796 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:16.936915 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:16.937094 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:16.938221 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:16.938231 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:16.938247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:16.938973 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:16.939075 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:16.939100 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:16.939211 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:16.939448 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:28:16.939750 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":377,"threads_started":1}
I20260811 06:28:16.942379 29501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:16.943488 29501 log.cc:824] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:16.945977 29501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: No bootstrap required, opened a new log
I20260811 06:28:16.952927 29501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } }
I20260811 06:28:16.953272 29501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:16.953392 29501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 013894e21f574b3c8ae103533c88e4db, State: Initialized, Role: FOLLOWER
I20260811 06:28:16.953972 29501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } }
I20260811 06:28:16.954245 29501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:16.954385 29501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:16.954584 29501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:16.956768 29501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } }
I20260811 06:28:16.957278 29501 leader_election.cc:302] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 013894e21f574b3c8ae103533c88e4db; no voters:
I20260811 06:28:16.957875 29501 leader_election.cc:288] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:16.958048 29506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:16.958818 29506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [term 1 LEADER]: Becoming Leader. State: Replica: 013894e21f574b3c8ae103533c88e4db, State: Running, Role: LEADER
I20260811 06:28:16.959378 29506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } }
I20260811 06:28:16.959928 29501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:16.962185 29508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 013894e21f574b3c8ae103533c88e4db. Latest consensus state: current_term: 1 leader_uuid: "013894e21f574b3c8ae103533c88e4db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } } }
I20260811 06:28:16.962620 29508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:16.963840 29507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "013894e21f574b3c8ae103533c88e4db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "013894e21f574b3c8ae103533c88e4db" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46349 } } }
I20260811 06:28:16.964179 29507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:16.968400 29516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:16.973347 29516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:16.982419 29516 catalog_manager.cc:1347] Generated new cluster ID: 8c3f3f916d81404da94cae094cae9f6f
I20260811 06:28:16.982543 29516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:17.000981 29516 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:17.002179 29516 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:17.018646 29516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: Generated new TSK 0
I20260811 06:28:17.019443 29516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:17.027936 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.129:0
--local_ip_for_outbound_sockets=127.7.4.129
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--builtin_ntp_servers=127.7.4.148:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:28:17.408674 29528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:17.408923 29528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:17.409094 29528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:17.416131 29528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:17.416245 29528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:17.416436 29528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.129
I20260811 06:28:17.425109 29528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.4.129
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.129
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:17.426306 29528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:17.428084 29528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:17.440747 29534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:17.440755 29535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:17.440835 29537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:17.441834 29528 server_base.cc:1034] running on GCE node
I20260811 06:28:17.442353 29528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:17.443171 29528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:17.444375 29528 hybrid_clock.cc:648] HybridClock initialized: now 1786429697444340 us; error 31 us; skew 500 ppm
I20260811 06:28:17.444643 29528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:17.451136 29528 webserver.cc:460] Webserver started at http://127.7.4.129:33615/ using document root <none> and password file <none>
I20260811 06:28:17.451849 29528 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:17.451970 29528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:17.452324 29528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:17.455073 29528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data/instance:
uuid: "778c76fa93744ec085ac2f311a73a7d5"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "0d97181afdcbdbf71c7c8beb732f1a18"
server_key_iv: "9da5b5cb99498f58ee83a36ce4f43c41"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.455935 29528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance:
uuid: "778c76fa93744ec085ac2f311a73a7d5"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "0d97181afdcbdbf71c7c8beb732f1a18"
server_key_iv: "9da5b5cb99498f58ee83a36ce4f43c41"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.461043 29528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:28:17.464648 29543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.466022 29528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:28:17.466255 29528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal
uuid: "778c76fa93744ec085ac2f311a73a7d5"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "0d97181afdcbdbf71c7c8beb732f1a18"
server_key_iv: "9da5b5cb99498f58ee83a36ce4f43c41"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.466477 29528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:17.477572 29528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:17.478320 29528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:17.478574 29528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:17.479562 29528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:17.481019 29528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:17.481124 29528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.481209 29528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:17.481276 29528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.514905 29528 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.129:32827
I20260811 06:28:17.515012 29675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.129:32827 every 8 connection(s)
I20260811 06:28:17.516297 29528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/data/info.pb
I20260811 06:28:17.517323 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29528
I20260811 06:28:17.517498 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal/instance
I20260811 06:28:17.518078 7186 external_mini_cluster.cc:1428] Setting key 27bd3230d7e1f1dd3656a1c159053032
I20260811 06:28:17.521624 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.130:0
--local_ip_for_outbound_sockets=127.7.4.130
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--builtin_ntp_servers=127.7.4.148:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:17.530071 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:17.519118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:59533 (local address 127.7.4.190:46349)
0811 06:28:17.519583 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:17.519588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:17.520958 (+ 1370us) server_negotiation.cc:407] Connection header received
0811 06:28:17.522879 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:17.522888 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:17.522979 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:17.523102 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:17.524177 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.525298 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.526862 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.527575 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.527737 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:17.529099 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:17.529109 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:17.529117 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:17.529142 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:17.529198 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:17.529219 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:17.529320 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:17.529512 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:28:17.529627 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":261,"threads_started":1}
I20260811 06:28:17.530867 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:17.519762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46349 (local address 127.7.4.129:59533)
0811 06:28:17.520777 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:17.520830 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:17.522608 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:17.523338 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:17.523349 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:17.523458 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:17.524000 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:17.524016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.525467 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.525471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:17.526717 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:17.526730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.527866 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.527872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:17.528069 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:17.528782 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:17.528784 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:17.528820 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:17.528893 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:17.529322 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:17.529330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:17.529335 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:28:17.529467 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 06:28:17.529740 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":332,"threads_started":1}
I20260811 06:28:17.532394 29676 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46349
I20260811 06:28:17.532706 29676 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:17.533581 29676 heartbeater.cc:511] Master 127.7.4.190:46349 requested a full tablet report, sending...
I20260811 06:28:17.535773 29463 ts_manager.cc:194] Registered new tserver with Master: 778c76fa93744ec085ac2f311a73a7d5 (127.7.4.129:32827)
I20260811 06:28:17.537724 29463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.129:59533
W20260811 06:28:17.868144 29680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:17.868386 29680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:17.868467 29680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:17.875618 29680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:17.875746 29680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:17.875855 29680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.130
I20260811 06:28:17.884433 29680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.4.130
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.130
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:17.885519 29680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:17.887235 29680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:17.900103 29685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:17.900142 29688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:17.900156 29686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:17.901338 29680 server_base.cc:1034] running on GCE node
I20260811 06:28:17.901939 29680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:17.902806 29680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:17.904023 29680 hybrid_clock.cc:648] HybridClock initialized: now 1786429697903988 us; error 50 us; skew 500 ppm
I20260811 06:28:17.904320 29680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:17.906383 29680 webserver.cc:460] Webserver started at http://127.7.4.130:33759/ using document root <none> and password file <none>
I20260811 06:28:17.907015 29680 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:17.907159 29680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:17.907616 29680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:17.910326 29680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data/instance:
uuid: "d2cb2f9ae959409b88f50f65c510341e"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "96cddb1c6733a15e1e4a93f0d901dcac"
server_key_iv: "c438189ef6f9a81cb2e7d5d46e7481fe"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.911092 29680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance:
uuid: "d2cb2f9ae959409b88f50f65c510341e"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "96cddb1c6733a15e1e4a93f0d901dcac"
server_key_iv: "c438189ef6f9a81cb2e7d5d46e7481fe"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.915762 29680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:28:17.919589 29694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.920846 29680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:28:17.921031 29680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal
uuid: "d2cb2f9ae959409b88f50f65c510341e"
format_stamp: "Formatted at 2026-08-11 06:28:17 on dist-test-slave-07c2"
server_key: "96cddb1c6733a15e1e4a93f0d901dcac"
server_key_iv: "c438189ef6f9a81cb2e7d5d46e7481fe"
server_key_version: "encryptionkey@0"
I20260811 06:28:17.921238 29680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:17.938407 29680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:17.939131 29680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:17.939446 29680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:17.940398 29680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:17.941908 29680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:17.942037 29680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.942162 29680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:17.942257 29680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:17.974815 29680 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.130:34307
I20260811 06:28:17.974851 29825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.130:34307 every 8 connection(s)
I20260811 06:28:17.976166 29680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/data/info.pb
I20260811 06:28:17.977337 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29680
I20260811 06:28:17.977524 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal/instance
I20260811 06:28:17.978012 7186 external_mini_cluster.cc:1428] Setting key bce7f1364d198b743460b9daf32bf686
I20260811 06:28:17.981196 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.4.131:0
--local_ip_for_outbound_sockets=127.7.4.131
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--builtin_ntp_servers=127.7.4.148:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:28:17.988938 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:17.978787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.130:49975 (local address 127.7.4.190:46349)
0811 06:28:17.979316 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:17.979324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:17.980657 (+ 1333us) server_negotiation.cc:407] Connection header received
0811 06:28:17.982494 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:17.982499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:17.982579 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:17.982698 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:17.983823 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.984561 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.986101 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.986501 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.986637 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:17.988062 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:17.988075 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:17.988078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:17.988095 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:17.988126 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:17.988138 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:17.988236 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:17.988476 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 06:28:17.988586 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
I20260811 06:28:17.990001 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:17.979413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46349 (local address 127.7.4.130:49975)
0811 06:28:17.980512 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:17.980562 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:17.982231 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:17.982914 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:17.982921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:17.983006 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:17.983641 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:17.983659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.984743 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.984747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:17.985954 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:17.985966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:17.986723 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:17.986730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:17.986941 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:17.987665 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:17.987668 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:17.987779 (+ 111us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:17.987869 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:17.988238 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:17.988253 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:17.988262 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:28:17.988453 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 06:28:17.988814 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":360,"threads_started":1}
I20260811 06:28:17.991626 29826 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46349
I20260811 06:28:17.991895 29826 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:17.992631 29826 heartbeater.cc:511] Master 127.7.4.190:46349 requested a full tablet report, sending...
I20260811 06:28:17.994151 29463 ts_manager.cc:194] Registered new tserver with Master: d2cb2f9ae959409b88f50f65c510341e (127.7.4.130:34307)
I20260811 06:28:17.995364 29463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.130:49975
W20260811 06:28:18.316392 29829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:18.316645 29829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:18.316811 29829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:18.323523 29829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:18.323626 29829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:18.323827 29829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.4.131
I20260811 06:28:18.332499 29829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.4.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.4.131
--webserver_port=0
--tserver_master_addrs=127.7.4.190:46349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.4.131
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:18.333704 29829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:18.335371 29829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:18.347651 29834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:18.347882 29838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:18.347857 29835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:18.349185 29829 server_base.cc:1034] running on GCE node
I20260811 06:28:18.349809 29829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:18.350732 29829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:18.351933 29829 hybrid_clock.cc:648] HybridClock initialized: now 1786429698351894 us; error 32 us; skew 500 ppm
I20260811 06:28:18.352244 29829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:28:18.354259 29829 webserver.cc:460] Webserver started at http://127.7.4.131:42331/ using document root <none> and password file <none>
I20260811 06:28:18.354894 29829 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:18.355014 29829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:18.355412 29829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:18.358134 29829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data/instance:
uuid: "8b432ae6cae3454ca4492e66c951e0f3"
format_stamp: "Formatted at 2026-08-11 06:28:18 on dist-test-slave-07c2"
server_key: "fa018f142d2c0588fe49d9e9150ef3d1"
server_key_iv: "e14ddf96e9d3deb34857a59db038d2ee"
server_key_version: "encryptionkey@0"
I20260811 06:28:18.358912 29829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance:
uuid: "8b432ae6cae3454ca4492e66c951e0f3"
format_stamp: "Formatted at 2026-08-11 06:28:18 on dist-test-slave-07c2"
server_key: "fa018f142d2c0588fe49d9e9150ef3d1"
server_key_iv: "e14ddf96e9d3deb34857a59db038d2ee"
server_key_version: "encryptionkey@0"
I20260811 06:28:18.363934 29829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 06:28:18.367863 29845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:18.369349 29829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:28:18.369508 29829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal
uuid: "8b432ae6cae3454ca4492e66c951e0f3"
format_stamp: "Formatted at 2026-08-11 06:28:18 on dist-test-slave-07c2"
server_key: "fa018f142d2c0588fe49d9e9150ef3d1"
server_key_iv: "e14ddf96e9d3deb34857a59db038d2ee"
server_key_version: "encryptionkey@0"
I20260811 06:28:18.369731 29829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:18.389647 29829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:18.390375 29829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:18.390643 29829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:18.391568 29829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:28:18.392920 29829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:28:18.393028 29829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:18.393151 29829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:28:18.393239 29829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:18.425498 29829 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.131:33275
I20260811 06:28:18.425575 29971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.131:33275 every 8 connection(s)
I20260811 06:28:18.426669 29829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/data/info.pb
I20260811 06:28:18.435144 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 29829
I20260811 06:28:18.435333 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal/instance
I20260811 06:28:18.435793 7186 external_mini_cluster.cc:1428] Setting key d02ba53e07062fa2d463f3c33f24d9fb
I20260811 06:28:18.438752 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.429208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.131:51711 (local address 127.7.4.190:46349)
0811 06:28:18.429393 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.429398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.430683 (+ 1285us) server_negotiation.cc:407] Connection header received
0811 06:28:18.432449 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.432454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.432517 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.432606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:18.433516 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.434231 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.435869 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.436272 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.436409 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.437914 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:18.437919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:18.437922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:18.437938 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:18.437969 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:18.437980 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:18.438051 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.438309 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.438386 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:28:18.439791 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.429667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46349 (local address 127.7.4.131:51711)
0811 06:28:18.430541 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.430582 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.432229 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.432778 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.432785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.432860 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:18.433382 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.433398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.434352 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.434356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.435727 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.435739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.436493 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.436501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.436809 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.437589 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:28:18.437597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:18.437633 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:28:18.437727 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:18.438100 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:18.438110 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:18.438118 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.438294 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.438778 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":297,"threads_started":1}
I20260811 06:28:18.440956 29972 heartbeater.cc:347] Connected to a master server at 127.7.4.190:46349
I20260811 06:28:18.441260 29972 heartbeater.cc:464] Registering TS with master...
I20260811 06:28:18.441956 29972 heartbeater.cc:511] Master 127.7.4.190:46349 requested a full tablet report, sending...
I20260811 06:28:18.443234 29463 ts_manager.cc:194] Registered new tserver with Master: 8b432ae6cae3454ca4492e66c951e0f3 (127.7.4.131:33275)
I20260811 06:28:18.444072 29463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.4.131:51711
I20260811 06:28:18.450953 7186 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:28:18.460597 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.454917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.7.4.190:46349)
0811 06:28:18.455060 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.455064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.455737 (+ 673us) server_negotiation.cc:407] Connection header received
0811 06:28:18.455811 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.455814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.455882 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.455967 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:18.456399 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.457044 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.458320 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.458835 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.458930 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.459995 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:18.459999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:18.460002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:18.460018 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:18.460046 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:18.460057 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:18.460123 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.460258 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.460355 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:28:18.464401 29463 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:28:18.470961 29463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:28:18.497328 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.482293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34307 (local address 127.0.0.1:34172)
0811 06:28:18.483328 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.483373 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.483563 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.489024 (+ 5461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.489031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.489067 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:18.489513 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.489532 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.491631 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.491639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.494481 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.494488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.496369 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.496376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.496593 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.496667 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.496766 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.496975 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":500,"thread_start_us":4903,"threads_started":1}
I20260811 06:28:18.497766 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.482675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.7.4.130:34307)
0811 06:28:18.488255 (+ 5580us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.488268 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.488304 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:28:18.488434 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.488445 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.488754 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.488998 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:18.489764 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.491477 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.494659 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.496143 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.496940 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.497168 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.497232 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.497405 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5298,"thread_start_us":5171,"threads_started":1}
I20260811 06:28:18.497788 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.482498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33275 (local address 127.0.0.1:54852)
0811 06:28:18.487484 (+ 4986us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.487504 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.487610 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.489190 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.489194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.489220 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:18.489680 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.489692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.491862 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.491867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.494103 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.494115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.496689 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.496695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.496975 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.497005 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.497099 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.497218 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4840,"mutex_wait_us":34,"thread_start_us":1079,"threads_started":1}
I20260811 06:28:18.498081 29987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.483152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.7.4.131:33275)
0811 06:28:18.488662 (+ 5510us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.488668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.488695 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:28:18.488766 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.488770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.488954 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.489894 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:18.490186 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.491704 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.494718 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.496482 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.496616 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.496886 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.497164 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.497353 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5324,"thread_start_us":5172,"threads_started":1}
I20260811 06:28:18.501669 29750 tablet_service.cc:1467] Processing CreateTablet for tablet e78c45a76fc9444f8701f92064bb1361 (DEFAULT_TABLE table=test-table [id=c250102d1c414a27b68ae70681958aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:18.501578 29898 tablet_service.cc:1467] Processing CreateTablet for tablet e78c45a76fc9444f8701f92064bb1361 (DEFAULT_TABLE table=test-table [id=c250102d1c414a27b68ae70681958aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:18.503008 29750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78c45a76fc9444f8701f92064bb1361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:18.503036 29898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78c45a76fc9444f8701f92064bb1361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:18.505079 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.489471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.129:32827 (local address 127.0.0.1:51082)
0811 06:28:18.495808 (+ 6337us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.495824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.495941 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.496680 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.496685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.496711 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:18.497609 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.497620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.499705 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.499711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.501690 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.501708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.504263 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.504288 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.504465 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.504506 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.504607 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.504735 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6199,"mutex_wait_us":1,"thread_start_us":6066,"threads_started":1}
I20260811 06:28:18.505174 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.487867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.7.4.129:32827)
0811 06:28:18.495501 (+ 7634us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.495512 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.496067 (+ 555us) server_negotiation.cc:407] Connection header received
0811 06:28:18.496171 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.496179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.496436 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.497841 (+ 1405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:18.498090 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.499541 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.501908 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.504050 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.504183 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.504428 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.504664 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.504851 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7467,"thread_start_us":7341,"threads_started":1}
I20260811 06:28:18.508671 29597 tablet_service.cc:1467] Processing CreateTablet for tablet e78c45a76fc9444f8701f92064bb1361 (DEFAULT_TABLE table=test-table [id=c250102d1c414a27b68ae70681958aca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:28:18.509974 29597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78c45a76fc9444f8701f92064bb1361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:18.521085 29991 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Bootstrap starting.
I20260811 06:28:18.524215 29992 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Bootstrap starting.
I20260811 06:28:18.525956 29991 tablet_bootstrap.cc:654] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:18.527307 29991 log.cc:824] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:18.528505 29992 tablet_bootstrap.cc:654] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:18.529105 29993 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Bootstrap starting.
I20260811 06:28:18.529825 29992 log.cc:824] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:18.532692 29992 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: No bootstrap required, opened a new log
I20260811 06:28:18.532964 29992 ts_tablet_manager.cc:1397] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 06:28:18.533100 29991 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: No bootstrap required, opened a new log
I20260811 06:28:18.533089 29993 tablet_bootstrap.cc:654] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:18.533361 29991 ts_tablet_manager.cc:1397] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 06:28:18.534341 29993 log.cc:824] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:18.538013 29993 tablet_bootstrap.cc:492] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: No bootstrap required, opened a new log
I20260811 06:28:18.538287 29993 ts_tablet_manager.cc:1397] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:28:18.540639 29676 heartbeater.cc:503] Master 127.7.4.190:46349 was elected leader, sending a full tablet report...
I20260811 06:28:18.541030 29992 raft_consensus.cc:348] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.541460 29992 raft_consensus.cc:374] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:18.541554 29992 raft_consensus.cc:729] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cb2f9ae959409b88f50f65c510341e, State: Initialized, Role: FOLLOWER
I20260811 06:28:18.542181 29992 consensus_queue.cc:260] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.545759 29991 raft_consensus.cc:348] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.546414 29991 raft_consensus.cc:374] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:18.546574 29991 raft_consensus.cc:729] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b432ae6cae3454ca4492e66c951e0f3, State: Initialized, Role: FOLLOWER
I20260811 06:28:18.547052 29826 heartbeater.cc:503] Master 127.7.4.190:46349 was elected leader, sending a full tablet report...
I20260811 06:28:18.547221 29991 consensus_queue.cc:260] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.547895 29992 ts_tablet_manager.cc:1428] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260811 06:28:18.549561 29993 raft_consensus.cc:348] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.550058 29972 heartbeater.cc:503] Master 127.7.4.190:46349 was elected leader, sending a full tablet report...
I20260811 06:28:18.550246 29993 raft_consensus.cc:374] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:18.550365 29993 raft_consensus.cc:729] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 778c76fa93744ec085ac2f311a73a7d5, State: Initialized, Role: FOLLOWER
I20260811 06:28:18.550709 29991 ts_tablet_manager.cc:1428] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
I20260811 06:28:18.551020 29993 consensus_queue.cc:260] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.552932 29993 ts_tablet_manager.cc:1428] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 06:28:18.567222 29999 raft_consensus.cc:482] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:28:18.567551 29999 raft_consensus.cc:504] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.569380 29999 leader_election.cc:288] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2cb2f9ae959409b88f50f65c510341e (127.7.4.130:34307), 8b432ae6cae3454ca4492e66c951e0f3 (127.7.4.131:33275)
I20260811 06:28:18.576331 29987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.569913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45751 (local address 127.7.4.131:33275)
0811 06:28:18.570118 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.570126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.570783 (+ 657us) server_negotiation.cc:407] Connection header received
0811 06:28:18.571089 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.571093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.571144 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.571235 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:18.571923 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.572691 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.574662 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.575466 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.575601 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.575705 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.575973 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.576091 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:28:18.576464 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.569817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.4.129:45043 (local address 127.7.4.130:34307)
0811 06:28:18.570058 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.570066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.570746 (+ 680us) server_negotiation.cc:407] Connection header received
0811 06:28:18.570989 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.570993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.571049 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.571141 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:28:18.571865 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.572619 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.574760 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.575560 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.575697 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.575792 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.576074 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.576204 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 06:28:18.576560 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.569831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.131:33275 (local address 127.7.4.129:45751)
0811 06:28:18.570654 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.570672 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.570943 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.571255 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.571260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.571320 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:18.571745 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.571755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.572837 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.572844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.574476 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.574488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.575640 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.575646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.575807 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.575830 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.575947 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.576071 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":2,"thread_start_us":515,"threads_started":1}
I20260811 06:28:18.576939 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.569779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.130:34307 (local address 127.7.4.129:45043)
0811 06:28:18.570619 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:18.570655 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:18.570812 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:18.571171 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:18.571175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:18.571197 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:28:18.571706 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.571722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.572749 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.572757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.574575 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:18.574587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.575765 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.575771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:18.575924 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.575946 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:28:18.576046 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:28:18.576160 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":89,"spinlock_wait_cycles":14720,"thread_start_us":480,"threads_started":1}
I20260811 06:28:18.577489 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78c45a76fc9444f8701f92064bb1361" candidate_uuid: "778c76fa93744ec085ac2f311a73a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b432ae6cae3454ca4492e66c951e0f3" is_pre_election: true
I20260811 06:28:18.577987 29923 raft_consensus.cc:2454] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 778c76fa93744ec085ac2f311a73a7d5 in term 0.
I20260811 06:28:18.577939 29774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78c45a76fc9444f8701f92064bb1361" candidate_uuid: "778c76fa93744ec085ac2f311a73a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cb2f9ae959409b88f50f65c510341e" is_pre_election: true
I20260811 06:28:18.578451 29774 raft_consensus.cc:2454] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 778c76fa93744ec085ac2f311a73a7d5 in term 0.
I20260811 06:28:18.578608 29545 leader_election.cc:302] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 778c76fa93744ec085ac2f311a73a7d5, 8b432ae6cae3454ca4492e66c951e0f3; no voters:
I20260811 06:28:18.578934 29999 raft_consensus.cc:2781] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:28:18.579092 29999 raft_consensus.cc:482] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:28:18.579205 29999 raft_consensus.cc:3037] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:18.581341 29999 raft_consensus.cc:504] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.581974 29999 leader_election.cc:288] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [CANDIDATE]: Term 1 election: Requested vote from peers d2cb2f9ae959409b88f50f65c510341e (127.7.4.130:34307), 8b432ae6cae3454ca4492e66c951e0f3 (127.7.4.131:33275)
I20260811 06:28:18.582386 29774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78c45a76fc9444f8701f92064bb1361" candidate_uuid: "778c76fa93744ec085ac2f311a73a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cb2f9ae959409b88f50f65c510341e"
I20260811 06:28:18.582507 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78c45a76fc9444f8701f92064bb1361" candidate_uuid: "778c76fa93744ec085ac2f311a73a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b432ae6cae3454ca4492e66c951e0f3"
I20260811 06:28:18.582561 29774 raft_consensus.cc:3037] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:18.582711 29923 raft_consensus.cc:3037] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:18.585230 29923 raft_consensus.cc:2454] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 778c76fa93744ec085ac2f311a73a7d5 in term 1.
I20260811 06:28:18.585230 29774 raft_consensus.cc:2454] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 778c76fa93744ec085ac2f311a73a7d5 in term 1.
I20260811 06:28:18.585631 29548 leader_election.cc:302] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 778c76fa93744ec085ac2f311a73a7d5, d2cb2f9ae959409b88f50f65c510341e; no voters:
I20260811 06:28:18.585924 29999 raft_consensus.cc:2781] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:18.586300 29999 raft_consensus.cc:686] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 1 LEADER]: Becoming Leader. State: Replica: 778c76fa93744ec085ac2f311a73a7d5, State: Running, Role: LEADER
I20260811 06:28:18.586798 29999 consensus_queue.cc:237] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } }
I20260811 06:28:18.591353 29462 catalog_manager.cc:5495] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 778c76fa93744ec085ac2f311a73a7d5 (127.7.4.129). New cstate: current_term: 1 leader_uuid: "778c76fa93744ec085ac2f311a73a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "778c76fa93744ec085ac2f311a73a7d5" member_type: VOTER last_known_addr { host: "127.7.4.129" port: 32827 } health_report { overall_health: HEALTHY } } }
I20260811 06:28:18.614354 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.609504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.7.4.129:32827)
0811 06:28:18.609686 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.609691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.609767 (+ 76us) server_negotiation.cc:407] Connection header received
0811 06:28:18.609943 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.609946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.610026 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.610146 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:28:18.610456 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.611120 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.612535 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.612938 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.613026 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.613431 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:28:18.613875 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:28:18.613976 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.614074 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.614150 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:28:18.621768 29923 raft_consensus.cc:1261] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 1 FOLLOWER]: Refusing update from remote peer 778c76fa93744ec085ac2f311a73a7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:18.621891 29774 raft_consensus.cc:1261] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 1 FOLLOWER]: Refusing update from remote peer 778c76fa93744ec085ac2f311a73a7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:18.622717 30002 consensus_queue.cc:1035] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b432ae6cae3454ca4492e66c951e0f3" member_type: VOTER last_known_addr { host: "127.7.4.131" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:18.623066 29999 consensus_queue.cc:1035] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cb2f9ae959409b88f50f65c510341e" member_type: VOTER last_known_addr { host: "127.7.4.130" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:18.633860 30005 mvcc.cc:205] Tried to move back new op lower bound from 7317216045546348544 to 7317216045416206336. Current Snapshot: MvccSnapshot[applied={T|T < 7317216045546348544}]
I20260811 06:28:18.636467 30008 mvcc.cc:205] Tried to move back new op lower bound from 7317216045546348544 to 7317216045416206336. Current Snapshot: MvccSnapshot[applied={T|T < 7317216045546348544}]
I20260811 06:28:18.636561 30007 mvcc.cc:205] Tried to move back new op lower bound from 7317216045546348544 to 7317216045416206336. Current Snapshot: MvccSnapshot[applied={T|T < 7317216045546348544}]
I20260811 06:28:18.649755 29462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:18.650031 29462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:28:18.653736 29462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 013894e21f574b3c8ae103533c88e4db: Sending DeleteTablet for 3 replicas of tablet e78c45a76fc9444f8701f92064bb1361
I20260811 06:28:18.654857 29898 tablet_service.cc:1514] Processing DeleteTablet for tablet e78c45a76fc9444f8701f92064bb1361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:18 UTC) from {username='slave'} at 127.0.0.1:54852
I20260811 06:28:18.654853 29597 tablet_service.cc:1514] Processing DeleteTablet for tablet e78c45a76fc9444f8701f92064bb1361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:18 UTC) from {username='slave'} at 127.0.0.1:51082
I20260811 06:28:18.654913 29750 tablet_service.cc:1514] Processing DeleteTablet for tablet e78c45a76fc9444f8701f92064bb1361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:28:18 UTC) from {username='slave'} at 127.0.0.1:34172
I20260811 06:28:18.655730 30017 tablet_replica.cc:330] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: stopping tablet replica
I20260811 06:28:18.656128 30017 raft_consensus.cc:2229] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:18.656545 30017 raft_consensus.cc:2258] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:18.658138 30017 ts_tablet_manager.cc:1905] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:18.660175 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29528
I20260811 06:28:18.665827 30017 ts_tablet_manager.cc:1918] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:18.666034 30017 log.cc:1196] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-2/wal/wals/e78c45a76fc9444f8701f92064bb1361
I20260811 06:28:18.666100 30019 tablet_replica.cc:330] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: stopping tablet replica
I20260811 06:28:18.666703 30017 ts_tablet_manager.cc:1939] T e78c45a76fc9444f8701f92064bb1361 P 8b432ae6cae3454ca4492e66c951e0f3: Deleting consensus metadata
I20260811 06:28:18.667042 30019 raft_consensus.cc:2229] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:28:18.668057 30019 raft_consensus.cc:2258] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:18.668833 29443 catalog_manager.cc:4841] TS 8b432ae6cae3454ca4492e66c951e0f3 (127.7.4.131:33275): tablet e78c45a76fc9444f8701f92064bb1361 (table test-table [id=c250102d1c414a27b68ae70681958aca]) successfully deleted
I20260811 06:28:18.669392 30018 tablet_replica.cc:330] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: stopping tablet replica
I20260811 06:28:18.669811 30018 raft_consensus.cc:2229] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:28:18.670603 30018 raft_consensus.cc:2258] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:28:18.672739 30019 ts_tablet_manager.cc:1905] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:18.672899 30018 ts_tablet_manager.cc:1905] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:28:18.679689 30019 ts_tablet_manager.cc:1918] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:18.679920 30019 log.cc:1196] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-1/wal/wals/e78c45a76fc9444f8701f92064bb1361
I20260811 06:28:18.680534 30019 ts_tablet_manager.cc:1939] T e78c45a76fc9444f8701f92064bb1361 P d2cb2f9ae959409b88f50f65c510341e: Deleting consensus metadata
I20260811 06:28:18.680874 30018 ts_tablet_manager.cc:1918] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:28:18.681102 30018 log.cc:1196] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Deleting WAL directory at /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429373903770-7186-0/minicluster-data/ts-0/wal/wals/e78c45a76fc9444f8701f92064bb1361
I20260811 06:28:18.681317 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.662784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.7.4.129:32827)
0811 06:28:18.663235 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.663243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.672394 (+ 9151us) server_negotiation.cc:407] Connection header received
0811 06:28:18.672915 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.672924 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.673050 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.673135 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:18.673943 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.675171 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.677192 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.677888 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.678007 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.679539 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:18.679551 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:18.679574 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:18.680366 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:18.680425 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:18.680454 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:18.680541 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.680804 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.680915 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260811 06:28:18.681798 30018 ts_tablet_manager.cc:1939] T e78c45a76fc9444f8701f92064bb1361 P 778c76fa93744ec085ac2f311a73a7d5: Deleting consensus metadata
I20260811 06:28:18.682323 29447 catalog_manager.cc:4841] TS d2cb2f9ae959409b88f50f65c510341e (127.7.4.130:34307): tablet e78c45a76fc9444f8701f92064bb1361 (table test-table [id=c250102d1c414a27b68ae70681958aca]) successfully deleted
I20260811 06:28:18.682850 29669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:18.683290 29447 catalog_manager.cc:4841] TS 778c76fa93744ec085ac2f311a73a7d5 (127.7.4.129:32827): tablet e78c45a76fc9444f8701f92064bb1361 (table test-table [id=c250102d1c414a27b68ae70681958aca]) successfully deleted
I20260811 06:28:18.778182 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29528
I20260811 06:28:18.800449 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29680
I20260811 06:28:18.807277 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.801222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.7.4.130:34307)
0811 06:28:18.801410 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.801418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.801439 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:28:18.801655 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.801660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.801737 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.801852 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:18.802259 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.802990 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.804416 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.804837 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.804975 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.806018 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:18.806027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:18.806040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:18.806580 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:18.806630 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:18.806649 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:18.806783 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.806878 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.806982 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:28:18.808076 29821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:18.895787 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29680
I20260811 06:28:18.917163 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29829
I20260811 06:28:18.924336 29987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:18.918004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.7.4.131:33275)
0811 06:28:18.918250 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:18.918256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:18.918277 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:28:18.918464 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:18.918468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:18.918553 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:18.918672 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:18.919096 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.919882 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.921177 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:18.921547 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:18.921678 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:18.922815 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:18.922825 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:18.922840 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:18.923644 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:18.923690 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:18.923709 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:18.923831 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:18.923955 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 06:28:18.924052 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:28:18.925004 29967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:19.012753 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29829
I20260811 06:28:19.035069 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 29426
I20260811 06:28:19.036178 29496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:19.145102 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 29426
2026-08-11T06:28:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2740 ms)
[----------] 6 tests from EncryptionPolicyTest (22238 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:19 dist-test-slave-07c2 krb5kdc[30027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:19 dist-test-slave-07c2 krb5kdc[30027](info): set up 2 sockets
Aug 11 06:28:19 dist-test-slave-07c2 krb5kdc[30027](info): commencing operation
krb5kdc: starting...
W20260811 06:28:20.332204 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:20 dist-test-slave-07c2 krb5kdc[30027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:20.627454 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:20.964145 30045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:20.964392 30045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:20.964478 30045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:20.971334 30045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:20.971441 30045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:20.971486 30045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:20.971551 30045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:20.971586 30045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:20.980553 30045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:20.981660 30045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:20.982834 30045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:20.994010 30050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:20.994089 30051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:20.994086 30053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:20.995172 30045 server_base.cc:1034] running on GCE node
I20260811 06:28:20.995893 30045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:20.997077 30045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:20.998271 30045 hybrid_clock.cc:648] HybridClock initialized: now 1786429700998221 us; error 45 us; skew 500 ppm
I20260811 06:28:20.998610 30045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:20 dist-test-slave-07c2 krb5kdc[30027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:21.001819 30045 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:21.003134 30045 webserver.cc:460] Webserver started at http://127.0.0.1:41811/ using document root <none> and password file <none>
I20260811 06:28:21.003847 30045 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:21.003947 30045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:21.004180 30045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:21.006897 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "fc796905786840e2aaa26af44edbaf02"
format_stamp: "Formatted at 2026-08-11 06:28:21 on dist-test-slave-07c2"
server_key: "0c51b82f7558cc36620def72c1752062"
server_key_iv: "b73c801cf410721ff8469493d208d0a0"
server_key_version: "encryptionkey@0"
I20260811 06:28:21.007648 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "fc796905786840e2aaa26af44edbaf02"
format_stamp: "Formatted at 2026-08-11 06:28:21 on dist-test-slave-07c2"
server_key: "0c51b82f7558cc36620def72c1752062"
server_key_iv: "b73c801cf410721ff8469493d208d0a0"
server_key_version: "encryptionkey@0"
I20260811 06:28:21.012715 30045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:28:21.015970 30062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:21.017213 30045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:28:21.017403 30045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "fc796905786840e2aaa26af44edbaf02"
format_stamp: "Formatted at 2026-08-11 06:28:21 on dist-test-slave-07c2"
server_key: "0c51b82f7558cc36620def72c1752062"
server_key_iv: "b73c801cf410721ff8469493d208d0a0"
server_key_version: "encryptionkey@0"
I20260811 06:28:21.017596 30045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:21.030995 30045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:21.035202 30045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:21.035527 30045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:21.053202 30045 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40151
I20260811 06:28:21.053243 30122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40151 every 8 connection(s)
I20260811 06:28:21.054564 30045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:21.055739 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30045
I20260811 06:28:21.056000 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:21.056835 7186 external_mini_cluster.cc:1428] Setting key 267b92055f72e61c4827c558eb5f0a48
I20260811 06:28:21.059201 30123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:28:21 dist-test-slave-07c2 krb5kdc[30027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:28:21.072515 30123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02: Bootstrap starting.
I20260811 06:28:21.075889 30123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:21.076907 30123 log.cc:824] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:21.080096 30123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02: No bootstrap required, opened a new log
I20260811 06:28:21.080641 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:21.058827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51729 (local address 127.0.0.1:40151)
0811 06:28:21.059609 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:21.059620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:21.059659 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:28:21.061267 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:21.061288 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:21.061650 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:21.061979 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:21.063780 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:21.065195 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:21.067060 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:21.067791 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:21.067917 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:21.072834 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:21.072841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:21.072852 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:21.072894 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:21.075971 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:21.076853 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:21.076874 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:21.076888 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:21.077073 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:21.077634 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:21.077637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:21.077639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:21.078215 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:21.078523 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:21.078949 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 06:28:21.079340 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":391,"threads_started":1}
I20260811 06:28:21.084641 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:21.090368 30123 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } }
I20260811 06:28:21.090817 30123 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:21.090940 30123 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc796905786840e2aaa26af44edbaf02, State: Initialized, Role: FOLLOWER
I20260811 06:28:21.091626 30123 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } }
I20260811 06:28:21.091871 30123 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:21.092026 30123 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:21.092200 30123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:21.094893 30123 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } }
I20260811 06:28:21.095393 30123 leader_election.cc:302] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc796905786840e2aaa26af44edbaf02; no voters:
I20260811 06:28:21.096076 30123 leader_election.cc:288] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:21.096192 30134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:21.097234 30134 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [term 1 LEADER]: Becoming Leader. State: Replica: fc796905786840e2aaa26af44edbaf02, State: Running, Role: LEADER
I20260811 06:28:21.099570 30134 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } }
I20260811 06:28:21.099952 30123 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:21.102464 30136 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc796905786840e2aaa26af44edbaf02. Latest consensus state: current_term: 1 leader_uuid: "fc796905786840e2aaa26af44edbaf02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } } }
I20260811 06:28:21.102885 30136 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:21.104569 30139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:21.104840 30135 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc796905786840e2aaa26af44edbaf02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc796905786840e2aaa26af44edbaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40151 } } }
I20260811 06:28:21.105343 30135 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:21.106937 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:21.090477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36499 (local address 127.0.0.1:40151)
0811 06:28:21.090632 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:21.090638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:21.091074 (+ 436us) server_negotiation.cc:407] Connection header received
0811 06:28:21.091235 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:21.091240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:21.091328 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:21.091439 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:21.092969 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:21.094167 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:21.095585 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:21.096126 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:21.096269 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:21.099013 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:21.099020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:21.099025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:21.099064 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:21.102537 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:21.103594 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:21.103599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:21.103602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:21.103745 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:21.104352 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:21.104359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:21.104363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:21.104764 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:21.105049 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:21.106245 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 06:28:21.106462 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 06:28:21.109555 30139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:21.122296 30139 catalog_manager.cc:1347] Generated new cluster ID: 0e6e3bfccc244b0ebd08458f9f8a3c83
I20260811 06:28:21.122403 30139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:21.143985 30139 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:21.145169 30139 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:21.170507 30139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc796905786840e2aaa26af44edbaf02: Generated new TSK 0
I20260811 06:28:21.171339 30139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:21.183913 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30045
I20260811 06:28:21.185055 30118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:21.277293 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30045
W20260811 06:28:21.298480 30025 connection.cc:535] client connection to 127.0.0.1:40151 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40151): Connection reset by peer (error 104)
2026-08-11T06:28:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2124 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:21 dist-test-slave-07c2 krb5kdc[30159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:21 dist-test-slave-07c2 krb5kdc[30159](info): set up 2 sockets
Aug 11 06:28:21 dist-test-slave-07c2 krb5kdc[30159](info): commencing operation
krb5kdc: starting...
W20260811 06:28:22.477439 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:22 dist-test-slave-07c2 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:22.804394 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:23.137307 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:23.137681 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:23.137869 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:23.145005 30175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:23.145139 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:23.145257 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:23.145361 30175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:23.145447 30175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:23.154853 30175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:23.156107 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:23.157402 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:23.168638 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:23.168602 30186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:23.168674 30184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:23.169632 30175 server_base.cc:1034] running on GCE node
I20260811 06:28:23.170377 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:23.171785 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:23.172984 30175 hybrid_clock.cc:648] HybridClock initialized: now 1786429703172922 us; error 52 us; skew 500 ppm
I20260811 06:28:23.173326 30175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:23 dist-test-slave-07c2 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:23.179738 30175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:23.181120 30175 webserver.cc:460] Webserver started at http://127.0.0.1:42407/ using document root <none> and password file <none>
I20260811 06:28:23.181766 30175 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:23.181881 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:23.182139 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:23.185258 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "90b90dd256fe491d8bf6a134db19d1af"
format_stamp: "Formatted at 2026-08-11 06:28:23 on dist-test-slave-07c2"
server_key: "0b3f6afd18087eecbf64999abbc64ab4"
server_key_iv: "58c752e2af44412d938d700e823468e0"
server_key_version: "encryptionkey@0"
I20260811 06:28:23.186259 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "90b90dd256fe491d8bf6a134db19d1af"
format_stamp: "Formatted at 2026-08-11 06:28:23 on dist-test-slave-07c2"
server_key: "0b3f6afd18087eecbf64999abbc64ab4"
server_key_iv: "58c752e2af44412d938d700e823468e0"
server_key_version: "encryptionkey@0"
I20260811 06:28:23.191385 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:28:23.194782 30195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:23.196106 30175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:28:23.196285 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "90b90dd256fe491d8bf6a134db19d1af"
format_stamp: "Formatted at 2026-08-11 06:28:23 on dist-test-slave-07c2"
server_key: "0b3f6afd18087eecbf64999abbc64ab4"
server_key_iv: "58c752e2af44412d938d700e823468e0"
server_key_version: "encryptionkey@0"
I20260811 06:28:23.196468 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:23.212953 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:23.216960 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:23.217228 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:23.234994 30175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34097
I20260811 06:28:23.235044 30254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34097 every 8 connection(s)
I20260811 06:28:23.236261 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:23.239802 30255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:23.244076 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30175
I20260811 06:28:23.244266 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:23.244817 7186 external_mini_cluster.cc:1428] Setting key 211540d7322254c6954eb3b091ec609e
I20260811 06:28:23.252687 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af: Bootstrap starting.
I20260811 06:28:23.256150 30255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:23.257335 30255 log.cc:824] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:23.257841 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:23.246552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59595 (local address 127.0.0.1:34097)
0811 06:28:23.247363 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:23.247374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:23.247415 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:28:23.248967 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:23.248986 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:23.249359 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:23.249666 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:23.250265 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:23.251583 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:23.253412 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:23.254041 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:23.254175 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:23.255506 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:23.255513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:23.255524 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:23.256021 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:23.256112 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:23.256138 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:23.256321 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:23.256580 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 06:28:23.256807 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":359,"threads_started":1}
I20260811 06:28:23.260357 30255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af: No bootstrap required, opened a new log
I20260811 06:28:23.261618 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:23.268544 30255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } }
I20260811 06:28:23.268985 30255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:23.269101 30255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90b90dd256fe491d8bf6a134db19d1af, State: Initialized, Role: FOLLOWER
I20260811 06:28:23.269650 30255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } }
I20260811 06:28:23.269892 30255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:23.270035 30255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:23.270258 30255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:23.272754 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:23.265902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38051 (local address 127.0.0.1:34097)
0811 06:28:23.266072 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:23.266077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:23.266608 (+ 531us) server_negotiation.cc:407] Connection header received
0811 06:28:23.266760 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:23.266765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:23.266832 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:23.266950 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:23.267436 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:23.268582 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:23.270222 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:23.270715 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:23.270868 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:23.271989 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:23.271997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:23.272005 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:23.272022 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:23.272074 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:23.272089 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:23.272208 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:23.272447 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 06:28:23.272498 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:28:23.273052 30255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } }
I20260811 06:28:23.273581 30255 leader_election.cc:302] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90b90dd256fe491d8bf6a134db19d1af; no voters:
I20260811 06:28:23.274204 30255 leader_election.cc:288] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:23.274302 30266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:23.274925 30266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [term 1 LEADER]: Becoming Leader. State: Replica: 90b90dd256fe491d8bf6a134db19d1af, State: Running, Role: LEADER
I20260811 06:28:23.275534 30266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } }
I20260811 06:28:23.276422 30255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:23.284667 30267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90b90dd256fe491d8bf6a134db19d1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } } }
I20260811 06:28:23.284725 30268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90b90dd256fe491d8bf6a134db19d1af. Latest consensus state: current_term: 1 leader_uuid: "90b90dd256fe491d8bf6a134db19d1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90b90dd256fe491d8bf6a134db19d1af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34097 } } }
I20260811 06:28:23.285032 30267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:23.285092 30268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:23.286839 30277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:23.291816 30277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:23.300236 30277 catalog_manager.cc:1347] Generated new cluster ID: 6ba9de3cc91341a7b796838bda8d4967
I20260811 06:28:23.300388 30277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:23.321973 30277 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:23.323673 30277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:23.329687 30277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90b90dd256fe491d8bf6a134db19d1af: Generated new TSK 0
I20260811 06:28:23.330454 30277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:23.338869 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30175
W20260811 06:28:23.340369 30250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59595
W20260811 06:28:23.341065 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30175: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:23.341259 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30175
W20260811 06:28:23.362507 30156 connection.cc:535] client connection to 127.0.0.1:34097 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34097): Connection reset by peer (error 104)
2026-08-11T06:28:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:23 dist-test-slave-07c2 krb5kdc[30287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:23 dist-test-slave-07c2 krb5kdc[30287](info): set up 2 sockets
Aug 11 06:28:23 dist-test-slave-07c2 krb5kdc[30287](info): commencing operation
krb5kdc: starting...
W20260811 06:28:24.520941 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:24 dist-test-slave-07c2 krb5kdc[30287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:24.824615 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:25.160933 30301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:25.161194 30301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:25.161314 30301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:25.168247 30301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:25.168408 30301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:25.168516 30301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:25.168639 30301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:25.168722 30301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:25.178232 30301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:25.179562 30301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:25.180943 30301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:25.192649 30307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:25.192793 30308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:25.192740 30310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:25.193755 30301 server_base.cc:1034] running on GCE node
I20260811 06:28:25.194486 30301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:25.195856 30301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:25.197078 30301 hybrid_clock.cc:648] HybridClock initialized: now 1786429705197035 us; error 35 us; skew 500 ppm
I20260811 06:28:25.197432 30301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:25 dist-test-slave-07c2 krb5kdc[30287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:25.200608 30301 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:25.202054 30301 webserver.cc:460] Webserver started at http://127.0.0.1:35593/ using document root <none> and password file <none>
I20260811 06:28:25.202653 30301 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:25.202754 30301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:25.202992 30301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:25.205858 30301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "cacdeb92a53d43eb87eb1f74539b93e6"
format_stamp: "Formatted at 2026-08-11 06:28:25 on dist-test-slave-07c2"
server_key: "9672187d6e825d9ac94628bf93eb8205"
server_key_iv: "4e09c52b66c1a5341262b3d194e0c372"
server_key_version: "encryptionkey@0"
I20260811 06:28:25.206589 30301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "cacdeb92a53d43eb87eb1f74539b93e6"
format_stamp: "Formatted at 2026-08-11 06:28:25 on dist-test-slave-07c2"
server_key: "9672187d6e825d9ac94628bf93eb8205"
server_key_iv: "4e09c52b66c1a5341262b3d194e0c372"
server_key_version: "encryptionkey@0"
I20260811 06:28:25.211606 30301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:28:25.214850 30317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:25.216204 30301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:25.216388 30301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "cacdeb92a53d43eb87eb1f74539b93e6"
format_stamp: "Formatted at 2026-08-11 06:28:25 on dist-test-slave-07c2"
server_key: "9672187d6e825d9ac94628bf93eb8205"
server_key_iv: "4e09c52b66c1a5341262b3d194e0c372"
server_key_version: "encryptionkey@0"
I20260811 06:28:25.216598 30301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:25.222497 30301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:25.226248 30301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:25.226534 30301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:25.244612 30301 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34195
I20260811 06:28:25.244695 30376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34195 every 8 connection(s)
I20260811 06:28:25.245944 30301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:25.249392 30377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:25.251899 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30301
I20260811 06:28:25.252080 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:25.252549 7186 external_mini_cluster.cc:1428] Setting key bc58325744a877b0e36c0295b9c1a82f
I20260811 06:28:25.261387 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:25.254324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38151 (local address 127.0.0.1:34195)
0811 06:28:25.255232 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:25.255248 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:25.255327 (+ 79us) server_negotiation.cc:407] Connection header received
0811 06:28:25.257653 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:25.257680 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:25.258082 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:25.258525 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:25.259498 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:25.259508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:25.259530 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:25.259547 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:25.259621 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:25.259649 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:25.259873 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:25.260013 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 06:28:25.260201 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":381,"threads_started":1}
I20260811 06:28:25.262722 30377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6: Bootstrap starting.
I20260811 06:28:25.265069 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:25.266108 30377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:25.267110 30377 log.cc:824] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:25.269624 30377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6: No bootstrap required, opened a new log
I20260811 06:28:25.273902 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:25.271436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57145 (local address 127.0.0.1:34195)
0811 06:28:25.271592 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:25.271599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:25.271950 (+ 351us) server_negotiation.cc:407] Connection header received
0811 06:28:25.272122 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:25.272127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:25.272181 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:25.272319 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:25.273272 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:25.273286 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:25.273289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:25.273303 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:25.273340 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:25.273357 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:25.273481 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:25.273586 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 06:28:25.273642 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:28:25.278430 30377 raft_consensus.cc:348] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260811 06:28:25.278789 30377 raft_consensus.cc:374] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:25.278896 30377 raft_consensus.cc:729] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacdeb92a53d43eb87eb1f74539b93e6, State: Initialized, Role: FOLLOWER
I20260811 06:28:25.279470 30377 consensus_queue.cc:260] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260811 06:28:25.279699 30377 raft_consensus.cc:388] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:25.279865 30377 raft_consensus.cc:482] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:25.280014 30377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:25.282172 30377 raft_consensus.cc:504] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260811 06:28:25.282620 30377 leader_election.cc:302] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cacdeb92a53d43eb87eb1f74539b93e6; no voters:
I20260811 06:28:25.283360 30377 leader_election.cc:288] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:25.283476 30389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:25.284300 30389 raft_consensus.cc:686] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [term 1 LEADER]: Becoming Leader. State: Replica: cacdeb92a53d43eb87eb1f74539b93e6, State: Running, Role: LEADER
I20260811 06:28:25.284830 30389 consensus_queue.cc:237] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260811 06:28:25.285537 30377 sys_catalog.cc:564] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:25.287823 30390 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260811 06:28:25.288076 30393 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cacdeb92a53d43eb87eb1f74539b93e6. Latest consensus state: current_term: 1 leader_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdeb92a53d43eb87eb1f74539b93e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260811 06:28:25.288338 30390 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:25.288396 30393 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:25.290586 30404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:25.295365 30404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:25.303396 30404 catalog_manager.cc:1347] Generated new cluster ID: d4d3e904e791459cbeb75cbfdc2e813b
I20260811 06:28:25.303496 30404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:25.340493 30404 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:25.340934 30404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:25.350225 30404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cacdeb92a53d43eb87eb1f74539b93e6: Generated new TSK 0
I20260811 06:28:25.351260 30404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:25.357153 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30301
W20260811 06:28:25.358716 30371 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38151
W20260811 06:28:25.359411 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30301: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:25.359575 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30301
2026-08-11T06:28:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:25 dist-test-slave-07c2 krb5kdc[30416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:25 dist-test-slave-07c2 krb5kdc[30416](info): set up 2 sockets
Aug 11 06:28:25 dist-test-slave-07c2 krb5kdc[30416](info): commencing operation
krb5kdc: starting...
W20260811 06:28:26.540663 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:26 dist-test-slave-07c2 krb5kdc[30416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:26.852844 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:27.196750 30430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:27.197055 30430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:27.197268 30430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:27.204752 30430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:27.204903 30430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:27.205015 30430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:27.205125 30430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:27.205212 30430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:27.215390 30430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:27.216794 30430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:27.218192 30430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:27.230206 30436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:27.230170 30437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:27.230252 30439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:27.231000 30430 server_base.cc:1034] running on GCE node
I20260811 06:28:27.231864 30430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:27.233256 30430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:27.234473 30430 hybrid_clock.cc:648] HybridClock initialized: now 1786429707234421 us; error 45 us; skew 500 ppm
I20260811 06:28:27.234804 30430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:27 dist-test-slave-07c2 krb5kdc[30416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:27.237869 30430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:27.239507 30430 webserver.cc:460] Webserver started at http://127.0.0.1:35495/ using document root <none> and password file <none>
I20260811 06:28:27.240126 30430 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:27.240243 30430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:27.240499 30430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:27.243386 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "782030315e84402e9aa84accd33265e7"
format_stamp: "Formatted at 2026-08-11 06:28:27 on dist-test-slave-07c2"
server_key: "20785e5c71b2969892d8ed7bfac0c050"
server_key_iv: "bf88098966e38d25e21b6d9cf3bcdba0"
server_key_version: "encryptionkey@0"
I20260811 06:28:27.244135 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "782030315e84402e9aa84accd33265e7"
format_stamp: "Formatted at 2026-08-11 06:28:27 on dist-test-slave-07c2"
server_key: "20785e5c71b2969892d8ed7bfac0c050"
server_key_iv: "bf88098966e38d25e21b6d9cf3bcdba0"
server_key_version: "encryptionkey@0"
I20260811 06:28:27.249090 30430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:28:27.252344 30447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:27.253597 30430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:28:27.253782 30430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "782030315e84402e9aa84accd33265e7"
format_stamp: "Formatted at 2026-08-11 06:28:27 on dist-test-slave-07c2"
server_key: "20785e5c71b2969892d8ed7bfac0c050"
server_key_iv: "bf88098966e38d25e21b6d9cf3bcdba0"
server_key_version: "encryptionkey@0"
I20260811 06:28:27.253970 30430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:27.272157 30430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:27.275821 30430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:27.276096 30430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:27.293653 30430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35767
I20260811 06:28:27.293686 30505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35767 every 8 connection(s)
I20260811 06:28:27.294953 30430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:27.298489 30506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:27.299715 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30430
I20260811 06:28:27.299926 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:27.300475 7186 external_mini_cluster.cc:1428] Setting key 0a5274765b98bcb2b8f2c751d0eaea7a
I20260811 06:28:27.312335 30506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7: Bootstrap starting.
Aug 11 06:28:27 dist-test-slave-07c2 krb5kdc[30416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:28:27.316098 30506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:27.317417 30506 log.cc:824] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:27.320681 30506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7: No bootstrap required, opened a new log
I20260811 06:28:27.324491 30509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:27.302219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48631 (local address 127.0.0.1:35767)
0811 06:28:27.303010 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:27.303021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:27.303062 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:28:27.304727 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:27.304748 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:27.305105 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:27.305423 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:27.307217 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:27.308517 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:27.310389 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:27.310991 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:27.311137 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:27.316152 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:27.316164 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:27.316182 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:27.316234 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:27.320162 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:27.320809 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:27.320817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:27.320826 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:27.321031 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:27.321416 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:27.321424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:27.321427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:27.322093 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:27.322392 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:27.322712 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 06:28:27.323132 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":383,"threads_started":1}
I20260811 06:28:27.328176 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:27.329896 30506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260811 06:28:27.330220 30506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:27.330317 30506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782030315e84402e9aa84accd33265e7, State: Initialized, Role: FOLLOWER
I20260811 06:28:27.330941 30506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260811 06:28:27.331249 30506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:27.331429 30506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:27.331614 30506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:27.334190 30506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260811 06:28:27.334645 30506 leader_election.cc:302] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782030315e84402e9aa84accd33265e7; no voters:
I20260811 06:28:27.335354 30506 leader_election.cc:288] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:27.335501 30516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:27.336756 30516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [term 1 LEADER]: Becoming Leader. State: Replica: 782030315e84402e9aa84accd33265e7, State: Running, Role: LEADER
I20260811 06:28:27.337294 30516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260811 06:28:27.337508 30506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:27.340008 30517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "782030315e84402e9aa84accd33265e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } } }
I20260811 06:28:27.340421 30517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:27.342844 30518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782030315e84402e9aa84accd33265e7. Latest consensus state: current_term: 1 leader_uuid: "782030315e84402e9aa84accd33265e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782030315e84402e9aa84accd33265e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } } }
I20260811 06:28:27.343225 30518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:27.344522 30521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:27.352236 30509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:27.332797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39373 (local address 127.0.0.1:35767)
0811 06:28:27.332989 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:27.332994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:27.333384 (+ 390us) server_negotiation.cc:407] Connection header received
0811 06:28:27.333498 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:27.333503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:27.333572 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:27.333669 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:27.335248 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:27.336525 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:27.338393 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:27.339179 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:27.342933 (+ 3754us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:27.343167 (+ 234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:27.343175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:27.343179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:27.343217 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:27.347203 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:27.348081 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:27.348086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:27.348089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:27.348744 (+ 655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:27.349284 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:27.349294 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:27.349300 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:27.349805 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:27.351329 (+ 1524us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:27.351521 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:28:27.351786 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:28:27.354250 30521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:27.371928 30521 catalog_manager.cc:1347] Generated new cluster ID: 24c1c005eae94d118d8c28133dd51c49
I20260811 06:28:27.372107 30521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:27.388932 30521 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:27.390111 30521 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:27.400063 30521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 782030315e84402e9aa84accd33265e7: Generated new TSK 0
I20260811 06:28:27.400888 30521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:27.408622 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30430
I20260811 06:28:27.409649 30501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:27.502343 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30430
2026-08-11T06:28:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:27 dist-test-slave-07c2 krb5kdc[30540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:27 dist-test-slave-07c2 krb5kdc[30540](info): set up 2 sockets
Aug 11 06:28:27 dist-test-slave-07c2 krb5kdc[30540](info): commencing operation
krb5kdc: starting...
W20260811 06:28:28.681978 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:28 dist-test-slave-07c2 krb5kdc[30540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:29.010613 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:29.343428 30554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:29.343717 30554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:29.343811 30554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:29.350775 30554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:29.350874 30554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:29.350927 30554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:29.350980 30554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:29.351017 30554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:29.360497 30554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:29.361687 30554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:29.362946 30554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:29.374619 30562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:29.374677 30561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:29.374799 30564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:29.375336 30554 server_base.cc:1034] running on GCE node
I20260811 06:28:29.376195 30554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:29.377566 30554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:29.378800 30554 hybrid_clock.cc:648] HybridClock initialized: now 1786429709378760 us; error 53 us; skew 500 ppm
I20260811 06:28:29.379175 30554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:29 dist-test-slave-07c2 krb5kdc[30540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429709, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:29.386658 30554 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:29.388314 30554 webserver.cc:460] Webserver started at http://127.0.0.1:44087/ using document root <none> and password file <none>
I20260811 06:28:29.389086 30554 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:29.389253 30554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:29.389569 30554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:29.392750 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "5f296dfabd644025bf21b0215840348d"
format_stamp: "Formatted at 2026-08-11 06:28:29 on dist-test-slave-07c2"
server_key: "23e05f865d564487fd94e4591828f0b3"
server_key_iv: "f176b90678284345c8051318182d4420"
server_key_version: "encryptionkey@0"
I20260811 06:28:29.393550 30554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "5f296dfabd644025bf21b0215840348d"
format_stamp: "Formatted at 2026-08-11 06:28:29 on dist-test-slave-07c2"
server_key: "23e05f865d564487fd94e4591828f0b3"
server_key_iv: "f176b90678284345c8051318182d4420"
server_key_version: "encryptionkey@0"
I20260811 06:28:29.398947 30554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:28:29.402421 30573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:29.403807 30554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:28:29.404006 30554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "5f296dfabd644025bf21b0215840348d"
format_stamp: "Formatted at 2026-08-11 06:28:29 on dist-test-slave-07c2"
server_key: "23e05f865d564487fd94e4591828f0b3"
server_key_iv: "f176b90678284345c8051318182d4420"
server_key_version: "encryptionkey@0"
I20260811 06:28:29.404202 30554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:29.425829 30554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:29.430228 30554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:29.430486 30554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:29.448302 30554 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35289
I20260811 06:28:29.448328 30629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35289 every 8 connection(s)
I20260811 06:28:29.449699 30554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:29.453059 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30554
I20260811 06:28:29.453275 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:29.453452 30630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:29.453787 7186 external_mini_cluster.cc:1428] Setting key 09ca75ac777c6eadd7bece733202da99
I20260811 06:28:29.468910 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:29.455561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41561 (local address 127.0.0.1:35289)
0811 06:28:29.456392 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:29.456407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:29.456462 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:28:29.458575 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:29.458602 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:29.459120 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:29.459581 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:29.460120 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:29.462112 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:29.463668 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:29.464728 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:29.464920 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:29.466131 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:29.466145 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:29.466162 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:29.466909 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:29.467014 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:29.467043 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:29.467159 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:29.467371 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:28:29.467622 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":337,"threads_started":1}
I20260811 06:28:29.469051 30630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d: Bootstrap starting.
I20260811 06:28:29.472643 30630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:29.472680 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:29.473640 30630 log.cc:824] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:29.476608 30630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d: No bootstrap required, opened a new log
I20260811 06:28:29.484400 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:29.477517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35937 (local address 127.0.0.1:35289)
0811 06:28:29.477706 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:29.477712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:29.478104 (+ 392us) server_negotiation.cc:407] Connection header received
0811 06:28:29.478238 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:29.478243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:29.478318 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:29.478448 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:29.478958 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:29.480098 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:29.481761 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:29.482499 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:29.482630 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:29.483684 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:29.483691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:29.483696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:29.483716 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:29.483764 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:29.483784 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:29.483891 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:29.483988 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 06:28:29.484076 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:28:29.486413 30630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260811 06:28:29.486724 30630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:29.486799 30630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f296dfabd644025bf21b0215840348d, State: Initialized, Role: FOLLOWER
I20260811 06:28:29.487253 30630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260811 06:28:29.487564 30630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:29.487668 30630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:29.487798 30630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:29.490002 30630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260811 06:28:29.490463 30630 leader_election.cc:302] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f296dfabd644025bf21b0215840348d; no voters:
I20260811 06:28:29.491003 30630 leader_election.cc:288] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:29.491170 30640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:29.492878 30640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [term 1 LEADER]: Becoming Leader. State: Replica: 5f296dfabd644025bf21b0215840348d, State: Running, Role: LEADER
I20260811 06:28:29.493496 30630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:29.493402 30640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260811 06:28:29.496418 30642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f296dfabd644025bf21b0215840348d. Latest consensus state: current_term: 1 leader_uuid: "5f296dfabd644025bf21b0215840348d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260811 06:28:29.496811 30642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:29.497339 30641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f296dfabd644025bf21b0215840348d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f296dfabd644025bf21b0215840348d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260811 06:28:29.497648 30641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:29.497717 30649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:29.503438 30649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:29.513204 30649 catalog_manager.cc:1347] Generated new cluster ID: 43b6f3a18ec64d79899a2706f4555751
I20260811 06:28:29.513381 30649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:29.540064 30649 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:29.541432 30649 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:29.557721 30649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f296dfabd644025bf21b0215840348d: Generated new TSK 0
I20260811 06:28:29.558650 30649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:29.569370 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30554
W20260811 06:28:29.570483 30625 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41561
W20260811 06:28:29.571209 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30554: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:29.571385 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30554
2026-08-11T06:28:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2074 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:29 dist-test-slave-07c2 krb5kdc[30668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:29 dist-test-slave-07c2 krb5kdc[30668](info): set up 2 sockets
Aug 11 06:28:29 dist-test-slave-07c2 krb5kdc[30668](info): commencing operation
krb5kdc: starting...
W20260811 06:28:30.752701 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:30 dist-test-slave-07c2 krb5kdc[30668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:31.094539 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:31.447121 30683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:31.447496 30683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:31.447661 30683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:31.454835 30683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:31.454995 30683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:31.455138 30683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:31.455243 30683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:31.455353 30683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:31.465050 30683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:31.466323 30683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:31.467741 30683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:31.479347 30690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:31.479358 30691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:31.479391 30693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:31.480244 30683 server_base.cc:1034] running on GCE node
I20260811 06:28:31.481074 30683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:31.482429 30683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:31.483656 30683 hybrid_clock.cc:648] HybridClock initialized: now 1786429711483613 us; error 54 us; skew 500 ppm
I20260811 06:28:31.484014 30683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:31 dist-test-slave-07c2 krb5kdc[30668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:31.487172 30683 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:31.488732 30683 webserver.cc:460] Webserver started at http://127.0.0.1:32773/ using document root <none> and password file <none>
I20260811 06:28:31.489570 30683 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:31.489706 30683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:31.490016 30683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:31.492991 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "a26de6efa37c426096d18b940d77aa22"
format_stamp: "Formatted at 2026-08-11 06:28:31 on dist-test-slave-07c2"
server_key: "2f87a8b70e4ae539b3afef433f6fddad"
server_key_iv: "022caff4bb2d3f1bcc26b96e3b06a30c"
server_key_version: "encryptionkey@0"
I20260811 06:28:31.493863 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "a26de6efa37c426096d18b940d77aa22"
format_stamp: "Formatted at 2026-08-11 06:28:31 on dist-test-slave-07c2"
server_key: "2f87a8b70e4ae539b3afef433f6fddad"
server_key_iv: "022caff4bb2d3f1bcc26b96e3b06a30c"
server_key_version: "encryptionkey@0"
I20260811 06:28:31.498911 30683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:28:31.502236 30701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:31.503443 30683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:28:31.503612 30683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "a26de6efa37c426096d18b940d77aa22"
format_stamp: "Formatted at 2026-08-11 06:28:31 on dist-test-slave-07c2"
server_key: "2f87a8b70e4ae539b3afef433f6fddad"
server_key_iv: "022caff4bb2d3f1bcc26b96e3b06a30c"
server_key_version: "encryptionkey@0"
I20260811 06:28:31.503795 30683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:31.509161 30683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:31.513360 30683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:31.513664 30683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:31.531414 30683 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32793
I20260811 06:28:31.531497 30760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32793 every 8 connection(s)
I20260811 06:28:31.532727 30683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:31.536370 30761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:31.538509 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30683
I20260811 06:28:31.538700 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:31.539160 7186 external_mini_cluster.cc:1428] Setting key 05ad829d2460cf139985c5691545f787
I20260811 06:28:31.548102 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:31.541062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60209 (local address 127.0.0.1:32793)
0811 06:28:31.541901 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:31.541916 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:31.541973 (+ 57us) server_negotiation.cc:407] Connection header received
0811 06:28:31.544351 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:31.544379 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:31.544764 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:31.545201 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:31.546162 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:31.546176 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:31.546192 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:31.546209 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:31.546290 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:31.546319 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:31.546546 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:31.546678 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:28:31.546874 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":342,"threads_started":1}
I20260811 06:28:31.550006 30761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22: Bootstrap starting.
I20260811 06:28:31.551859 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:31.553295 30761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:31.554271 30761 log.cc:824] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:31.557488 30761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22: No bootstrap required, opened a new log
I20260811 06:28:31.558758 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:31.556158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48263 (local address 127.0.0.1:32793)
0811 06:28:31.556365 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:31.556374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:31.556823 (+ 449us) server_negotiation.cc:407] Connection header received
0811 06:28:31.556993 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:31.556997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:31.557040 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:31.557118 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:31.558109 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:31.558112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:31.558115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:31.558130 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:31.558173 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:31.558187 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:31.558272 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:31.558431 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 06:28:31.558473 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:28:31.565095 30761 raft_consensus.cc:348] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } }
I20260811 06:28:31.565498 30761 raft_consensus.cc:374] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:31.565625 30761 raft_consensus.cc:729] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26de6efa37c426096d18b940d77aa22, State: Initialized, Role: FOLLOWER
I20260811 06:28:31.566321 30761 consensus_queue.cc:260] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } }
I20260811 06:28:31.566538 30761 raft_consensus.cc:388] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:31.566638 30761 raft_consensus.cc:482] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:31.566756 30761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:31.568840 30761 raft_consensus.cc:504] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } }
I20260811 06:28:31.569243 30761 leader_election.cc:302] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a26de6efa37c426096d18b940d77aa22; no voters:
I20260811 06:28:31.569743 30761 leader_election.cc:288] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:31.569890 30771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:31.570544 30771 raft_consensus.cc:686] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [term 1 LEADER]: Becoming Leader. State: Replica: a26de6efa37c426096d18b940d77aa22, State: Running, Role: LEADER
I20260811 06:28:31.571075 30771 consensus_queue.cc:237] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } }
I20260811 06:28:31.571756 30761 sys_catalog.cc:564] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:31.577112 30772 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a26de6efa37c426096d18b940d77aa22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } } }
I20260811 06:28:31.577541 30772 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:31.580482 30778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:31.580451 30773 sys_catalog.cc:455] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a26de6efa37c426096d18b940d77aa22. Latest consensus state: current_term: 1 leader_uuid: "a26de6efa37c426096d18b940d77aa22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a26de6efa37c426096d18b940d77aa22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32793 } } }
I20260811 06:28:31.580745 30773 sys_catalog.cc:458] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:31.586544 30778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:31.596710 30778 catalog_manager.cc:1347] Generated new cluster ID: 816b84497b0545dfb1f6c305ce004cf8
I20260811 06:28:31.596819 30778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:31.641258 30778 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:31.641741 30778 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:31.663369 30778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a26de6efa37c426096d18b940d77aa22: Generated new TSK 0
I20260811 06:28:31.664194 30778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:31.677554 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30683
W20260811 06:28:31.678627 30755 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60209
W20260811 06:28:31.679373 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30683: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:31.679522 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30683
2026-08-11T06:28:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2103 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:31 dist-test-slave-07c2 krb5kdc[30796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:31 dist-test-slave-07c2 krb5kdc[30796](info): set up 2 sockets
Aug 11 06:28:31 dist-test-slave-07c2 krb5kdc[30796](info): commencing operation
krb5kdc: starting...
W20260811 06:28:32.865096 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.008s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:33.168560 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:33.516929 30811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:33.517218 30811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:33.517377 30811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:33.525060 30811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:33.525218 30811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:33.525314 30811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:33.525432 30811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:33.525497 30811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:33.535638 30811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:33.537045 30811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:33.538414 30811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:33.551000 30816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:33.550905 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:33.550935 30818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:33.551952 30811 server_base.cc:1034] running on GCE node
I20260811 06:28:33.552732 30811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:33.554251 30811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:33.555455 30811 hybrid_clock.cc:648] HybridClock initialized: now 1786429713555393 us; error 56 us; skew 500 ppm
I20260811 06:28:33.555822 30811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:33.559134 30811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:33.560781 30811 webserver.cc:460] Webserver started at http://127.0.0.1:39811/ using document root <none> and password file <none>
I20260811 06:28:33.561501 30811 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:33.561640 30811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:33.561975 30811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:33.565114 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "beb78ce339d74ba4b5b93c47aca734fb"
format_stamp: "Formatted at 2026-08-11 06:28:33 on dist-test-slave-07c2"
server_key: "56854321c822ebfd565812a498518085"
server_key_iv: "1bae1eb40e56f3975d9dac1423d85c96"
server_key_version: "encryptionkey@0"
I20260811 06:28:33.565922 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "beb78ce339d74ba4b5b93c47aca734fb"
format_stamp: "Formatted at 2026-08-11 06:28:33 on dist-test-slave-07c2"
server_key: "56854321c822ebfd565812a498518085"
server_key_iv: "1bae1eb40e56f3975d9dac1423d85c96"
server_key_version: "encryptionkey@0"
I20260811 06:28:33.571506 30811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 06:28:33.577307 30829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:33.579046 30811 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:28:33.579211 30811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "beb78ce339d74ba4b5b93c47aca734fb"
format_stamp: "Formatted at 2026-08-11 06:28:33 on dist-test-slave-07c2"
server_key: "56854321c822ebfd565812a498518085"
server_key_iv: "1bae1eb40e56f3975d9dac1423d85c96"
server_key_version: "encryptionkey@0"
I20260811 06:28:33.579459 30811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:33.594684 30811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:33.598614 30811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:33.598870 30811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:33.616633 30811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43779
I20260811 06:28:33.616667 30883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43779 every 8 connection(s)
I20260811 06:28:33.618091 30811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:33.621800 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:33.627993 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30811
I20260811 06:28:33.628191 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:33.628675 7186 external_mini_cluster.cc:1428] Setting key 7caf690be208c1d77c72388eb27baaaf
I20260811 06:28:33.632885 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb: Bootstrap starting.
I20260811 06:28:33.636700 30884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:33.637846 30884 log.cc:824] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:33.641124 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb: No bootstrap required, opened a new log
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:28:33.650358 30884 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260811 06:28:33.650786 30884 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:33.650903 30884 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb78ce339d74ba4b5b93c47aca734fb, State: Initialized, Role: FOLLOWER
I20260811 06:28:33.651646 30884 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260811 06:28:33.651895 30884 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:33.652064 30884 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:33.652254 30884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:33.653019 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:33.630718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:45447 (local address 127.0.0.1:43779)
0811 06:28:33.631548 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:33.631560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:33.631601 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:28:33.633346 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:33.633368 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:33.633757 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:33.634078 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:33.636112 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:33.637472 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:33.639453 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:33.640113 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:33.640318 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:33.645069 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:33.645077 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:33.645093 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:33.645128 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:33.648357 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:33.649303 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:33.649309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:33.649314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:33.649478 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:33.649972 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:33.649975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:33.649977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:33.650468 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:33.650731 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:33.651382 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 06:28:33.651748 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":374,"threads_started":1}
I20260811 06:28:33.654702 30884 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260811 06:28:33.655187 30884 leader_election.cc:302] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb78ce339d74ba4b5b93c47aca734fb; no voters:
I20260811 06:28:33.655880 30884 leader_election.cc:288] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:33.656038 30889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:33.657150 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:33.658356 30884 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:33.662776 30889 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [term 1 LEADER]: Becoming Leader. State: Replica: beb78ce339d74ba4b5b93c47aca734fb, State: Running, Role: LEADER
I20260811 06:28:33.663576 30889 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260811 06:28:33.666890 30891 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb78ce339d74ba4b5b93c47aca734fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } } }
I20260811 06:28:33.667970 30891 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:33.667538 30892 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb78ce339d74ba4b5b93c47aca734fb. Latest consensus state: current_term: 1 leader_uuid: "beb78ce339d74ba4b5b93c47aca734fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb78ce339d74ba4b5b93c47aca734fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } } }
I20260811 06:28:33.668488 30892 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:33.682389 30908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:33.689575 30908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:33.697152 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:33.665237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52273 (local address 127.0.0.1:43779)
0811 06:28:33.665458 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:33.665464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:33.677403 (+ 11939us) server_negotiation.cc:407] Connection header received
0811 06:28:33.677916 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:33.677925 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:33.678026 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:33.678109 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:33.681226 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:33.682400 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:33.684254 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:33.685074 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:33.685221 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:33.688855 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:33.688862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:33.688867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:33.688905 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:33.692835 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:33.693853 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:33.693859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:33.693862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:33.694036 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:33.694678 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:33.694685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:33.694688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:33.695165 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:33.695488 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:33.696586 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 06:28:33.696781 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:28:33.700954 30908 catalog_manager.cc:1347] Generated new cluster ID: 6bf73b4978944b75b163f010d2963e59
I20260811 06:28:33.701059 30908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:33.715662 30908 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:33.716846 30908 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:33.723680 30908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb78ce339d74ba4b5b93c47aca734fb: Generated new TSK 0
I20260811 06:28:33.724426 30908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:33.731092 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30811
I20260811 06:28:33.732197 30879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:33.821462 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30811
W20260811 06:28:33.842790 30793 connection.cc:535] client connection to 127.0.0.1:43779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43779): Connection reset by peer (error 104)
2026-08-11T06:28:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2141 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30921](info): set up 2 sockets
Aug 11 06:28:33 dist-test-slave-07c2 krb5kdc[30921](info): commencing operation
krb5kdc: starting...
W20260811 06:28:35.024888 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:35 dist-test-slave-07c2 krb5kdc[30921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:35.325997 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:35.664037 30935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:35.664373 30935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:35.664525 30935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:35.671856 30935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:35.672015 30935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:35.672116 30935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:35.672222 30935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:35.672297 30935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:35.682699 30935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:35.684093 30935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:35.685426 30935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:35.697379 30940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:35.697422 30941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:35.697427 30943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:35.698474 30935 server_base.cc:1034] running on GCE node
I20260811 06:28:35.699230 30935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:35.700670 30935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:35.701907 30935 hybrid_clock.cc:648] HybridClock initialized: now 1786429715701870 us; error 52 us; skew 500 ppm
I20260811 06:28:35.702258 30935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:35 dist-test-slave-07c2 krb5kdc[30921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:35.708948 30935 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:35.710536 30935 webserver.cc:460] Webserver started at http://127.0.0.1:41345/ using document root <none> and password file <none>
I20260811 06:28:35.711233 30935 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:35.711418 30935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:35.711730 30935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:35.714684 30935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "3392b82e40b34b31a1d4da80141257a6"
format_stamp: "Formatted at 2026-08-11 06:28:35 on dist-test-slave-07c2"
server_key: "61970e26b3482dc5c17a47828be6f120"
server_key_iv: "fd71d97fadfef534cc0236debcb60e13"
server_key_version: "encryptionkey@0"
I20260811 06:28:35.715538 30935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "3392b82e40b34b31a1d4da80141257a6"
format_stamp: "Formatted at 2026-08-11 06:28:35 on dist-test-slave-07c2"
server_key: "61970e26b3482dc5c17a47828be6f120"
server_key_iv: "fd71d97fadfef534cc0236debcb60e13"
server_key_version: "encryptionkey@0"
I20260811 06:28:35.720718 30935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:28:35.723973 30951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:35.725301 30935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:28:35.725476 30935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "3392b82e40b34b31a1d4da80141257a6"
format_stamp: "Formatted at 2026-08-11 06:28:35 on dist-test-slave-07c2"
server_key: "61970e26b3482dc5c17a47828be6f120"
server_key_iv: "fd71d97fadfef534cc0236debcb60e13"
server_key_version: "encryptionkey@0"
I20260811 06:28:35.725674 30935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:35.746695 30935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:35.751194 30935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:35.751513 30935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:35.770395 30935 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41869
I20260811 06:28:35.770430 31008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41869 every 8 connection(s)
I20260811 06:28:35.771802 30935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:35.775574 31010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:35.775899 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 30935
I20260811 06:28:35.776093 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:35.776575 7186 external_mini_cluster.cc:1428] Setting key 4bbd240c996207efeb506da8a1ccdb0a
I20260811 06:28:35.789278 31010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6: Bootstrap starting.
I20260811 06:28:35.790882 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:35.778363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52535 (local address 127.0.0.1:41869)
0811 06:28:35.779190 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:35.779201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:35.779242 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:28:35.781584 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:35.781627 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:35.782034 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:35.782368 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:35.783012 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:35.784379 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:35.786398 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:35.787086 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:35.787217 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:35.788571 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:35.788579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:35.788591 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:35.789130 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:35.789203 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:35.789226 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:35.789358 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:35.789739 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 06:28:35.789917 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":398,"threads_started":1}
I20260811 06:28:35.792603 31010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:35.793805 31010 log.cc:824] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:35.794772 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:35.797026 31010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6: No bootstrap required, opened a new log
I20260811 06:28:35.806792 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:35.800074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:37313 (local address 127.0.0.1:41869)
0811 06:28:35.800283 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:35.800290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:35.800721 (+ 431us) server_negotiation.cc:407] Connection header received
0811 06:28:35.800851 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:35.800856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:35.800941 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:35.801056 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:35.801545 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:35.802697 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:35.804037 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:35.804768 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:35.804910 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:35.805999 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:35.806005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:35.806011 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:35.806032 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:35.806092 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:35.806114 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:35.806237 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:35.806414 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:28:35.806469 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:35.806905 31010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } }
I20260811 06:28:35.807379 31010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:35.807536 31010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3392b82e40b34b31a1d4da80141257a6, State: Initialized, Role: FOLLOWER
I20260811 06:28:35.808203 31010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } }
I20260811 06:28:35.808463 31010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:35.808634 31010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:35.808821 31010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:35.810997 31010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } }
I20260811 06:28:35.811594 31010 leader_election.cc:302] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3392b82e40b34b31a1d4da80141257a6; no voters:
I20260811 06:28:35.812254 31010 leader_election.cc:288] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:35.812379 31021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:35.813452 31021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [term 1 LEADER]: Becoming Leader. State: Replica: 3392b82e40b34b31a1d4da80141257a6, State: Running, Role: LEADER
I20260811 06:28:35.813987 31021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } }
I20260811 06:28:35.814601 31010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:35.817014 31023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3392b82e40b34b31a1d4da80141257a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } } }
I20260811 06:28:35.817467 31023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:35.818598 31031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:35.816991 31024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3392b82e40b34b31a1d4da80141257a6. Latest consensus state: current_term: 1 leader_uuid: "3392b82e40b34b31a1d4da80141257a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3392b82e40b34b31a1d4da80141257a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41869 } } }
I20260811 06:28:35.819768 31024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:35.823859 31031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:35.834208 31031 catalog_manager.cc:1347] Generated new cluster ID: 55f33395c8a144169bcea0a106bd812c
I20260811 06:28:35.834342 31031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:35.845532 31031 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:35.846822 31031 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:35.856966 31031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3392b82e40b34b31a1d4da80141257a6: Generated new TSK 0
I20260811 06:28:35.857846 31031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:35.866219 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30935
W20260811 06:28:35.867443 31004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:52535
W20260811 06:28:35.868208 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 30935: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:35.868366 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 30935
W20260811 06:28:35.889707 30918 connection.cc:535] client connection to 127.0.0.1:41869 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41869): Connection reset by peer (error 104)
2026-08-11T06:28:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:35 dist-test-slave-07c2 krb5kdc[31044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:35 dist-test-slave-07c2 krb5kdc[31044](info): set up 2 sockets
Aug 11 06:28:35 dist-test-slave-07c2 krb5kdc[31044](info): commencing operation
krb5kdc: starting...
W20260811 06:28:37.056599 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:37 dist-test-slave-07c2 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:37.374804 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:37.716944 31060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:37.717206 31060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:37.717382 31060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:37.724444 31060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:37.724584 31060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:37.724696 31060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:37.724798 31060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:37.724875 31060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:37.734589 31060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:37.735816 31060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:37.737126 31060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:37.749043 31068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:37.749068 31066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:37.749328 31065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:37.749944 31060 server_base.cc:1034] running on GCE node
I20260811 06:28:37.750655 31060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:37.752076 31060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:37.753324 31060 hybrid_clock.cc:648] HybridClock initialized: now 1786429717753280 us; error 47 us; skew 500 ppm
I20260811 06:28:37.753655 31060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:37 dist-test-slave-07c2 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:37.756850 31060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:37.758169 31060 webserver.cc:460] Webserver started at http://127.0.0.1:35025/ using document root <none> and password file <none>
I20260811 06:28:37.758759 31060 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:37.758869 31060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:37.759114 31060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:37.761994 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "dbb710f9b62342519aa475a057801790"
format_stamp: "Formatted at 2026-08-11 06:28:37 on dist-test-slave-07c2"
server_key: "3d72049d3f673c83ebd2bf5e571e2317"
server_key_iv: "7ea343aa4d342bf6ce1345d65d5ca6b3"
server_key_version: "encryptionkey@0"
I20260811 06:28:37.762681 31060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "dbb710f9b62342519aa475a057801790"
format_stamp: "Formatted at 2026-08-11 06:28:37 on dist-test-slave-07c2"
server_key: "3d72049d3f673c83ebd2bf5e571e2317"
server_key_iv: "7ea343aa4d342bf6ce1345d65d5ca6b3"
server_key_version: "encryptionkey@0"
I20260811 06:28:37.767550 31060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:28:37.770726 31075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:37.771975 31060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:37.772113 31060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "dbb710f9b62342519aa475a057801790"
format_stamp: "Formatted at 2026-08-11 06:28:37 on dist-test-slave-07c2"
server_key: "3d72049d3f673c83ebd2bf5e571e2317"
server_key_iv: "7ea343aa4d342bf6ce1345d65d5ca6b3"
server_key_version: "encryptionkey@0"
I20260811 06:28:37.772264 31060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:37.777534 31060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:37.781618 31060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:37.781865 31060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:37.799779 31060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37923
I20260811 06:28:37.799857 31136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37923 every 8 connection(s)
I20260811 06:28:37.801043 31060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:37.804131 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31060
I20260811 06:28:37.804337 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:37.804731 31137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:37.804973 7186 external_mini_cluster.cc:1428] Setting key 17582eb7154d16a9c1f895747d34093d
I20260811 06:28:37.812763 31140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:37.806703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:34289 (local address 127.0.0.1:37923)
0811 06:28:37.807522 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:37.807557 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:37.807615 (+ 58us) server_negotiation.cc:407] Connection header received
0811 06:28:37.809397 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:37.809416 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:37.809709 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:37.810085 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:37.811107 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:37.811116 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:37.811127 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:37.811140 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:37.811189 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:37.811207 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:37.811501 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:37.811651 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:28:37.811871 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":340,"threads_started":1}
I20260811 06:28:37.816561 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:37.818110 31137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790: Bootstrap starting.
I20260811 06:28:37.821161 31137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:37.822345 31137 log.cc:824] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:37.823597 31140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:37.820921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:38135 (local address 127.0.0.1:37923)
0811 06:28:37.821131 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:37.821139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:37.821689 (+ 550us) server_negotiation.cc:407] Connection header received
0811 06:28:37.821802 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:37.821807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:37.821862 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:37.821969 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:37.822822 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:37.822834 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:37.822841 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:37.822867 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:37.822922 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:37.822943 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:37.823058 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:37.823208 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:28:37.823251 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 06:28:37.825323 31137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790: No bootstrap required, opened a new log
I20260811 06:28:37.832497 31137 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260811 06:28:37.832800 31137 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:37.832871 31137 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb710f9b62342519aa475a057801790, State: Initialized, Role: FOLLOWER
I20260811 06:28:37.833324 31137 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260811 06:28:37.833534 31137 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:37.833630 31137 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:37.833745 31137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:37.835762 31137 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260811 06:28:37.836162 31137 leader_election.cc:302] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb710f9b62342519aa475a057801790; no voters:
I20260811 06:28:37.836655 31137 leader_election.cc:288] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:37.836804 31149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:37.837508 31149 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [term 1 LEADER]: Becoming Leader. State: Replica: dbb710f9b62342519aa475a057801790, State: Running, Role: LEADER
I20260811 06:28:37.838038 31149 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } }
I20260811 06:28:37.838644 31137 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:37.841041 31150 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbb710f9b62342519aa475a057801790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } } }
I20260811 06:28:37.841471 31150 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:37.841324 31151 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb710f9b62342519aa475a057801790. Latest consensus state: current_term: 1 leader_uuid: "dbb710f9b62342519aa475a057801790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb710f9b62342519aa475a057801790" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37923 } } }
I20260811 06:28:37.841588 31151 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:37.844167 31158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:37.848351 31158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:37.856462 31158 catalog_manager.cc:1347] Generated new cluster ID: 298e2af2d1784815b1d739f95cbac448
I20260811 06:28:37.856570 31158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:37.872840 31158 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:37.873271 31158 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:37.883766 31158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbb710f9b62342519aa475a057801790: Generated new TSK 0
I20260811 06:28:37.884545 31158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:37.895752 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31060
W20260811 06:28:37.896927 31132 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:34289
W20260811 06:28:37.897603 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31060: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:37.897749 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31060
2026-08-11T06:28:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2029 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:38 dist-test-slave-07c2 krb5kdc[31173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:38 dist-test-slave-07c2 krb5kdc[31173](info): set up 2 sockets
Aug 11 06:28:38 dist-test-slave-07c2 krb5kdc[31173](info): commencing operation
krb5kdc: starting...
W20260811 06:28:39.081827 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:39 dist-test-slave-07c2 krb5kdc[31173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:39.406360 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:39.750825 31187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:39.751092 31187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:39.751332 31187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:39.758590 31187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:39.758692 31187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:39.758798 31187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:39.758900 31187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:39.758984 31187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:39.768831 31187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:39.770105 31187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:39.771461 31187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:39.783196 31195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:39.783229 31194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:39.783152 31197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:39.784161 31187 server_base.cc:1034] running on GCE node
I20260811 06:28:39.784905 31187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:39.786324 31187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:39.787583 31187 hybrid_clock.cc:648] HybridClock initialized: now 1786429719787523 us; error 42 us; skew 500 ppm
I20260811 06:28:39.788003 31187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:39 dist-test-slave-07c2 krb5kdc[31173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429719, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:39.791355 31187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:39.792802 31187 webserver.cc:460] Webserver started at http://127.0.0.1:43527/ using document root <none> and password file <none>
I20260811 06:28:39.793514 31187 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:39.793664 31187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:39.793982 31187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:39.797051 31187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0"
format_stamp: "Formatted at 2026-08-11 06:28:39 on dist-test-slave-07c2"
server_key: "06106e4f9284b77a1ccc06dcf624954c"
server_key_iv: "45ad976ead17eaf17e2c3b78998c94bd"
server_key_version: "encryptionkey@0"
I20260811 06:28:39.797853 31187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0"
format_stamp: "Formatted at 2026-08-11 06:28:39 on dist-test-slave-07c2"
server_key: "06106e4f9284b77a1ccc06dcf624954c"
server_key_iv: "45ad976ead17eaf17e2c3b78998c94bd"
server_key_version: "encryptionkey@0"
I20260811 06:28:39.803013 31187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:28:39.806378 31204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:39.807680 31187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:39.807864 31187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0"
format_stamp: "Formatted at 2026-08-11 06:28:39 on dist-test-slave-07c2"
server_key: "06106e4f9284b77a1ccc06dcf624954c"
server_key_iv: "45ad976ead17eaf17e2c3b78998c94bd"
server_key_version: "encryptionkey@0"
I20260811 06:28:39.808053 31187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:39.829851 31187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:39.834192 31187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:39.834496 31187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:39.852829 31187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37829
I20260811 06:28:39.852870 31263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37829 every 8 connection(s)
I20260811 06:28:39.854203 31187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:39.857172 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31187
I20260811 06:28:39.857366 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:39.857858 7186 external_mini_cluster.cc:1428] Setting key 2c3a4465b8ae9d5036e62cf6dc0ebf66
I20260811 06:28:39.858060 31264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:39.872225 31264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: Bootstrap starting.
Aug 11 06:28:39 dist-test-slave-07c2 krb5kdc[31173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:28:39.875962 31264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:39.877362 31264 log.cc:824] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:39.880302 31264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: No bootstrap required, opened a new log
I20260811 06:28:39.882534 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:39.859813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:55637 (local address 127.0.0.1:37829)
0811 06:28:39.860624 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:39.860653 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:39.860705 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:28:39.862969 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:39.862996 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:39.863510 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:39.863954 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:39.865212 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:39.867216 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:39.868509 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:39.869608 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:39.869808 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:39.874293 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:39.874303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:39.874320 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:39.874367 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:39.878410 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:39.879203 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:39.879215 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:39.879221 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:39.879446 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:39.879924 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:39.879929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:39.879933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:39.880440 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:39.880797 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:39.881304 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 06:28:39.881610 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":325,"threads_started":1}
I20260811 06:28:39.886404 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:39.888433 31264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260811 06:28:39.888804 31264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:39.888928 31264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a3c7a4c06ca46b7a1a1df350510b7e0, State: Initialized, Role: FOLLOWER
I20260811 06:28:39.889531 31264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260811 06:28:39.889779 31264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:39.889953 31264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:39.890144 31264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:39.892782 31264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260811 06:28:39.893270 31264 leader_election.cc:302] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a3c7a4c06ca46b7a1a1df350510b7e0; no voters:
I20260811 06:28:39.893991 31264 leader_election.cc:288] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:39.894104 31275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:39.896088 31264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:39.899590 31275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [term 1 LEADER]: Becoming Leader. State: Replica: 9a3c7a4c06ca46b7a1a1df350510b7e0, State: Running, Role: LEADER
I20260811 06:28:39.900326 31275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260811 06:28:39.903571 31276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } } }
I20260811 06:28:39.903980 31276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:39.904510 31277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a3c7a4c06ca46b7a1a1df350510b7e0. Latest consensus state: current_term: 1 leader_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3c7a4c06ca46b7a1a1df350510b7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } } }
I20260811 06:28:39.905004 31277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:39.909121 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:39.890908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:45939 (local address 127.0.0.1:37829)
0811 06:28:39.891093 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:39.891099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:39.891634 (+ 535us) server_negotiation.cc:407] Connection header received
0811 06:28:39.891746 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:39.891751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:39.891829 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:39.891932 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:39.893364 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:39.894704 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:39.896618 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:39.897615 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:39.898643 (+ 1028us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:39.901503 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:39.901509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:39.901514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:39.901555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:39.905811 (+ 4256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:39.906779 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:39.906788 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:39.906794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:39.906976 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:39.907570 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:39.907575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:39.907579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:39.907895 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:39.908245 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:39.908640 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 06:28:39.908813 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 06:28:39.911670 31291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:28:39.911798 31291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:28:39.911903 31289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:39.914073 31289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:39.922710 31289 catalog_manager.cc:1347] Generated new cluster ID: 98c965e4e3dc460b8875617baf3875f3
I20260811 06:28:39.922820 31289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:39.952643 31289 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:39.954852 31289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:39.963919 31289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a3c7a4c06ca46b7a1a1df350510b7e0: Generated new TSK 0
I20260811 06:28:39.964978 31289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:39.973244 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31187
I20260811 06:28:39.974925 31258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:40.067444 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31187
W20260811 06:28:40.089574 31171 connection.cc:535] client connection to 127.0.0.1:37829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37829): Connection reset by peer (error 104)
2026-08-11T06:28:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2174 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:40 dist-test-slave-07c2 krb5kdc[31300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:40 dist-test-slave-07c2 krb5kdc[31300](info): set up 2 sockets
Aug 11 06:28:40 dist-test-slave-07c2 krb5kdc[31300](info): commencing operation
krb5kdc: starting...
W20260811 06:28:41.249511 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:41 dist-test-slave-07c2 krb5kdc[31300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:41.603119 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:41.945537 31313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:41.945827 31313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:41.945986 31313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:41.952939 31313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:41.953035 31313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:41.953147 31313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:41.953263 31313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:41.953337 31313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:41.962944 31313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:41.964146 31313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:41.965400 31313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:41.976873 31325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:41.976835 31321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:41.976915 31319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:41.977963 31313 server_base.cc:1034] running on GCE node
I20260811 06:28:41.978704 31313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:41.980005 31313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:41.981209 31313 hybrid_clock.cc:648] HybridClock initialized: now 1786429721981171 us; error 41 us; skew 500 ppm
I20260811 06:28:41.981489 31313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:41 dist-test-slave-07c2 krb5kdc[31300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429721, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:41.989116 31313 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:41.990564 31313 webserver.cc:460] Webserver started at http://127.0.0.1:42517/ using document root <none> and password file <none>
I20260811 06:28:41.991212 31313 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:41.991392 31313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:41.991699 31313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:41.994547 31313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "5966b3d5559a4c63a58803f9c060c0f3"
format_stamp: "Formatted at 2026-08-11 06:28:41 on dist-test-slave-07c2"
server_key: "42fec8ea80d3f3b04d746740412fefb6"
server_key_iv: "aa80b324e2deb5998a717870d9108446"
server_key_version: "encryptionkey@0"
I20260811 06:28:41.995386 31313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "5966b3d5559a4c63a58803f9c060c0f3"
format_stamp: "Formatted at 2026-08-11 06:28:41 on dist-test-slave-07c2"
server_key: "42fec8ea80d3f3b04d746740412fefb6"
server_key_iv: "aa80b324e2deb5998a717870d9108446"
server_key_version: "encryptionkey@0"
I20260811 06:28:42.000308 31313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:28:42.003629 31333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:42.004879 31313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:42.005069 31313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "5966b3d5559a4c63a58803f9c060c0f3"
format_stamp: "Formatted at 2026-08-11 06:28:41 on dist-test-slave-07c2"
server_key: "42fec8ea80d3f3b04d746740412fefb6"
server_key_iv: "aa80b324e2deb5998a717870d9108446"
server_key_version: "encryptionkey@0"
I20260811 06:28:42.005259 31313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:42.045806 31313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:42.051962 31313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:42.052333 31313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:42.071851 31313 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40757
I20260811 06:28:42.071887 31386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40757 every 8 connection(s)
I20260811 06:28:42.073283 31313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:42.076957 31387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:42.077350 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31313
I20260811 06:28:42.077550 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:42.078161 7186 external_mini_cluster.cc:1428] Setting key 68d4e2c0aaf9d99a675e4d6a6b05c59c
I20260811 06:28:42.090436 31387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3: Bootstrap starting.
I20260811 06:28:42.092054 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:42.079952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:56451 (local address 127.0.0.1:40757)
0811 06:28:42.080804 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:42.080814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:42.080854 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:28:42.083045 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:42.083066 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:42.083519 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:42.083839 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:42.084423 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:42.085640 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:42.087544 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:42.088229 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:42.088359 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:42.089773 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:42.089781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:42.089793 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:42.090306 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:42.090388 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:42.090411 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:42.090559 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:42.090835 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 06:28:42.091038 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":416,"threads_started":1}
I20260811 06:28:42.093659 31387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:42.094825 31387 log.cc:824] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:42.095815 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:42.097519 31387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3: No bootstrap required, opened a new log
I20260811 06:28:42.107201 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:42.100448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:56803 (local address 127.0.0.1:40757)
0811 06:28:42.100595 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:42.100599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:42.101114 (+ 515us) server_negotiation.cc:407] Connection header received
0811 06:28:42.101285 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:42.101288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:42.101343 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:42.101417 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:42.102009 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:42.102746 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:42.104612 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:42.105080 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:42.105175 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:42.106399 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:42.106403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:42.106406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:42.106422 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:42.106458 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:42.106472 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:42.106551 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:42.106788 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:28:42.106828 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 06:28:42.106950 31387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260811 06:28:42.107517 31387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:42.107630 31387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5966b3d5559a4c63a58803f9c060c0f3, State: Initialized, Role: FOLLOWER
I20260811 06:28:42.108311 31387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260811 06:28:42.108656 31387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:42.108800 31387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:42.108980 31387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:42.111235 31387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260811 06:28:42.111773 31387 leader_election.cc:302] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5966b3d5559a4c63a58803f9c060c0f3; no voters:
I20260811 06:28:42.112365 31387 leader_election.cc:288] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:42.112520 31399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:42.113991 31399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5966b3d5559a4c63a58803f9c060c0f3, State: Running, Role: LEADER
I20260811 06:28:42.114497 31399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260811 06:28:42.114643 31387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:42.117355 31401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5966b3d5559a4c63a58803f9c060c0f3. Latest consensus state: current_term: 1 leader_uuid: "5966b3d5559a4c63a58803f9c060c0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } } }
I20260811 06:28:42.117355 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5966b3d5559a4c63a58803f9c060c0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5966b3d5559a4c63a58803f9c060c0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } } }
I20260811 06:28:42.117794 31401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:42.117834 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:42.119853 31407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:42.126024 31407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:42.135025 31407 catalog_manager.cc:1347] Generated new cluster ID: 325d5dd7a8b141fb96cf5b2fff838cde
I20260811 06:28:42.135135 31407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:42.152427 31407 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:42.153679 31407 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:42.168725 31407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5966b3d5559a4c63a58803f9c060c0f3: Generated new TSK 0
I20260811 06:28:42.169512 31407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:42.179828 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31313
W20260811 06:28:42.181028 31382 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:56451
W20260811 06:28:42.181830 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31313: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:42.181985 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31313
W20260811 06:28:42.203279 31297 connection.cc:535] client connection to 127.0.0.1:40757 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40757): Connection reset by peer (error 104)
2026-08-11T06:28:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:42 dist-test-slave-07c2 krb5kdc[31429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:42 dist-test-slave-07c2 krb5kdc[31429](info): set up 2 sockets
Aug 11 06:28:42 dist-test-slave-07c2 krb5kdc[31429](info): commencing operation
krb5kdc: starting...
W20260811 06:28:43.376430 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:43 dist-test-slave-07c2 krb5kdc[31429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:28:43.686475 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:44.026859 31445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:44.027143 31445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:44.027235 31445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:44.034478 31445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:44.034586 31445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:44.034634 31445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:44.034687 31445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:44.034724 31445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:44.044759 31445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:44.046046 31445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:44.047384 31445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:44.059405 31450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:44.059450 31453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:44.059701 31451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:44.060339 31445 server_base.cc:1034] running on GCE node
I20260811 06:28:44.061277 31445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:44.062752 31445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:44.064003 31445 hybrid_clock.cc:648] HybridClock initialized: now 1786429724063970 us; error 48 us; skew 500 ppm
I20260811 06:28:44.064368 31445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:44 dist-test-slave-07c2 krb5kdc[31429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:44.067731 31445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:28:44.069129 31445 webserver.cc:460] Webserver started at http://127.0.0.1:35725/ using document root <none> and password file <none>
I20260811 06:28:44.069766 31445 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:44.069866 31445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:44.070116 31445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:44.073062 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "8d248322a658483aad495e29b18745f5"
format_stamp: "Formatted at 2026-08-11 06:28:44 on dist-test-slave-07c2"
server_key: "e3f6e777dd33bafddcc4b72daa2ea885"
server_key_iv: "ed5fe0ea235695d7eb3e85ad4e1f2bca"
server_key_version: "encryptionkey@0"
I20260811 06:28:44.073855 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "8d248322a658483aad495e29b18745f5"
format_stamp: "Formatted at 2026-08-11 06:28:44 on dist-test-slave-07c2"
server_key: "e3f6e777dd33bafddcc4b72daa2ea885"
server_key_iv: "ed5fe0ea235695d7eb3e85ad4e1f2bca"
server_key_version: "encryptionkey@0"
I20260811 06:28:44.078770 31445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:28:44.082178 31464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:44.083576 31445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:44.083717 31445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "8d248322a658483aad495e29b18745f5"
format_stamp: "Formatted at 2026-08-11 06:28:44 on dist-test-slave-07c2"
server_key: "e3f6e777dd33bafddcc4b72daa2ea885"
server_key_iv: "ed5fe0ea235695d7eb3e85ad4e1f2bca"
server_key_version: "encryptionkey@0"
I20260811 06:28:44.083951 31445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:44.089329 31445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:44.093586 31445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:44.093834 31445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:44.112742 31445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39649
I20260811 06:28:44.112808 31524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39649 every 8 connection(s)
I20260811 06:28:44.114112 31445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:44.114634 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31445
I20260811 06:28:44.114842 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:44.115482 7186 external_mini_cluster.cc:1428] Setting key c9dccd5df71990d7f6ee9d07800482af
I20260811 06:28:44.118396 31525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:44.123265 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:44.117321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:44465 (local address 127.0.0.1:39649)
0811 06:28:44.118096 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:44.118108 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:44.118146 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:28:44.119906 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:44.119927 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:44.120229 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:44.120607 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:44.121718 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:44.121724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:44.121740 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:44.121755 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:44.121828 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:44.121856 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:44.122102 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:44.122234 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:28:44.122419 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":320,"threads_started":1}
I20260811 06:28:44.127096 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:44.130474 31525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5: Bootstrap starting.
I20260811 06:28:44.134104 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:44.131721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:53985 (local address 127.0.0.1:39649)
0811 06:28:44.131911 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:44.131920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:44.132252 (+ 332us) server_negotiation.cc:407] Connection header received
0811 06:28:44.132380 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:44.132384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:44.132446 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:44.132554 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:44.133471 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:44.133476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:44.133481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:44.133499 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:44.133549 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:44.133569 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:44.133693 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:44.133745 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:28:44.133804 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:28:44.134248 31525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:44.135633 31525 log.cc:824] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:44.138315 31525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5: No bootstrap required, opened a new log
I20260811 06:28:44.145432 31525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } }
I20260811 06:28:44.145787 31525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:44.145890 31525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d248322a658483aad495e29b18745f5, State: Initialized, Role: FOLLOWER
I20260811 06:28:44.146417 31525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } }
I20260811 06:28:44.146653 31525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:44.146798 31525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:44.146960 31525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:44.149137 31525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } }
I20260811 06:28:44.149699 31525 leader_election.cc:302] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d248322a658483aad495e29b18745f5; no voters:
I20260811 06:28:44.150329 31525 leader_election.cc:288] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:44.150468 31536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:44.151175 31536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [term 1 LEADER]: Becoming Leader. State: Replica: 8d248322a658483aad495e29b18745f5, State: Running, Role: LEADER
I20260811 06:28:44.151767 31536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } }
I20260811 06:28:44.152515 31525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:44.154989 31539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d248322a658483aad495e29b18745f5. Latest consensus state: current_term: 1 leader_uuid: "8d248322a658483aad495e29b18745f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } } }
I20260811 06:28:44.155017 31537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d248322a658483aad495e29b18745f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d248322a658483aad495e29b18745f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39649 } } }
I20260811 06:28:44.155508 31537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:44.155508 31539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:44.158219 31547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:44.163375 31547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:44.171813 31547 catalog_manager.cc:1347] Generated new cluster ID: 97b88c216d124718bf28c6ab60829035
I20260811 06:28:44.171912 31547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:44.191931 31547 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:44.192366 31547 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:44.198472 31547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d248322a658483aad495e29b18745f5: Generated new TSK 0
I20260811 06:28:44.199174 31547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:44.206866 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31445
W20260811 06:28:44.207926 31519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:44465
W20260811 06:28:44.208626 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:44.208786 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31445
2026-08-11T06:28:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:44 dist-test-slave-07c2 krb5kdc[31565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:44 dist-test-slave-07c2 krb5kdc[31565](info): set up 2 sockets
Aug 11 06:28:44 dist-test-slave-07c2 krb5kdc[31565](info): commencing operation
krb5kdc: starting...
W20260811 06:28:45.389356 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:45 dist-test-slave-07c2 krb5kdc[31565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:45.688768 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:38415
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:38415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:46.019528 31578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:46.019793 31578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:46.019974 31578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:46.026820 31578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:46.026939 31578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:46.027048 31578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:46.027139 31578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:46.027237 31578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:46.036773 31578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:38415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:38415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:46.037978 31578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:46.039202 31578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:46.050627 31583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:46.050570 31586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:46.050642 31584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:46.051695 31578 server_base.cc:1034] running on GCE node
I20260811 06:28:46.052405 31578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:46.053687 31578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:46.054883 31578 hybrid_clock.cc:648] HybridClock initialized: now 1786429726054844 us; error 31 us; skew 500 ppm
I20260811 06:28:46.055212 31578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:46 dist-test-slave-07c2 krb5kdc[31565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:46.058286 31578 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:46.059687 31578 webserver.cc:460] Webserver started at http://127.7.4.190:40273/ using document root <none> and password file <none>
I20260811 06:28:46.060328 31578 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:46.060479 31578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:46.060781 31578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:46.063666 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "cb5fe28c29f94f898ac4eb39209eebf3"
format_stamp: "Formatted at 2026-08-11 06:28:46 on dist-test-slave-07c2"
server_key: "28ef17a189bcb97fd59594d8df17bd37"
server_key_iv: "fd8938d304b9f5f65b4fb2411dd978ca"
server_key_version: "encryptionkey@0"
I20260811 06:28:46.064416 31578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "cb5fe28c29f94f898ac4eb39209eebf3"
format_stamp: "Formatted at 2026-08-11 06:28:46 on dist-test-slave-07c2"
server_key: "28ef17a189bcb97fd59594d8df17bd37"
server_key_iv: "fd8938d304b9f5f65b4fb2411dd978ca"
server_key_version: "encryptionkey@0"
I20260811 06:28:46.069711 31578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:28:46.073048 31593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:46.074222 31578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:28:46.074429 31578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "cb5fe28c29f94f898ac4eb39209eebf3"
format_stamp: "Formatted at 2026-08-11 06:28:46 on dist-test-slave-07c2"
server_key: "28ef17a189bcb97fd59594d8df17bd37"
server_key_iv: "fd8938d304b9f5f65b4fb2411dd978ca"
server_key_version: "encryptionkey@0"
I20260811 06:28:46.074633 31578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:46.097468 31578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:46.101327 31578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:46.101598 31578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:46.119880 31578 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:38415
I20260811 06:28:46.119920 31650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:38415 every 8 connection(s)
I20260811 06:28:46.121155 31578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:46.124449 31651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:46.126765 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31578
I20260811 06:28:46.126952 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:46.127540 7186 external_mini_cluster.cc:1428] Setting key 02c53d8ba3969355ffbfbef2f53d971d
I20260811 06:28:46.138785 31651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3: Bootstrap starting.
Aug 11 06:28:46 dist-test-slave-07c2 krb5kdc[31565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:46.141950 31651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:46.143188 31651 log.cc:824] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:46.145995 31651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3: No bootstrap required, opened a new log
I20260811 06:28:46.150532 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:46.129288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46131 (local address 127.7.4.190:38415)
0811 06:28:46.130203 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:46.130217 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:46.130266 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:28:46.131991 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:46.132018 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:46.132491 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:46.132878 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:46.134141 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:46.136149 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:46.137398 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:46.138421 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:46.138587 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:46.143101 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:46.143108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:46.143120 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:46.143168 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:46.146352 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:46.147250 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:46.147256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:46.147261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:46.147478 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:46.147971 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:46.147974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:46.147976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:46.148462 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:46.148711 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:46.149253 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 06:28:46.149541 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":415,"threads_started":1}
I20260811 06:28:46.154217 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:46.154537 31651 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } }
I20260811 06:28:46.154913 31651 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:46.155004 31651 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb5fe28c29f94f898ac4eb39209eebf3, State: Initialized, Role: FOLLOWER
I20260811 06:28:46.155551 31651 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } }
I20260811 06:28:46.155828 31651 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:46.155946 31651 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:46.156092 31651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:46.159267 31651 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } }
I20260811 06:28:46.159947 31651 leader_election.cc:302] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb5fe28c29f94f898ac4eb39209eebf3; no voters:
I20260811 06:28:46.160624 31651 leader_election.cc:288] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:46.160681 31663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:46.162263 31663 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [term 1 LEADER]: Becoming Leader. State: Replica: cb5fe28c29f94f898ac4eb39209eebf3, State: Running, Role: LEADER
I20260811 06:28:46.162806 31663 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } }
I20260811 06:28:46.162978 31651 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:46.165319 31665 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb5fe28c29f94f898ac4eb39209eebf3. Latest consensus state: current_term: 1 leader_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } } }
I20260811 06:28:46.165699 31665 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:46.166260 31664 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5fe28c29f94f898ac4eb39209eebf3" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 38415 } } }
I20260811 06:28:46.166595 31664 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:46.172838 31668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:46.181394 31668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:46.182082 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:46.159387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55441 (local address 127.7.4.190:38415)
0811 06:28:46.159591 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:46.159597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:46.159968 (+ 371us) server_negotiation.cc:407] Connection header received
0811 06:28:46.160141 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:46.160156 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:46.160226 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:46.160304 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:46.162377 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:46.163568 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:46.165803 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:46.166764 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:46.166885 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:46.169806 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:46.169815 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:46.169823 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:46.169868 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:46.176797 (+ 6929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:46.177962 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:46.177967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:46.177971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:46.178124 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:46.178804 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:46.178811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:46.178821 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:46.179229 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:46.181015 (+ 1786us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:46.181267 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 06:28:46.181506 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:28:46.192502 31668 catalog_manager.cc:1347] Generated new cluster ID: 4289fef5596e4a638c1ddc722444589e
I20260811 06:28:46.192638 31668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:46.208870 31668 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:46.210490 31668 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:46.217684 31668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb5fe28c29f94f898ac4eb39209eebf3: Generated new TSK 0
I20260811 06:28:46.218637 31668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:46.230746 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31578
I20260811 06:28:46.232611 31645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:46.323009 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31578
W20260811 06:28:46.345062 31562 connection.cc:535] client connection to 127.7.4.190:38415 recv error: Network error: failed to read from TLS socket (remote: 127.7.4.190:38415): Connection reset by peer (error 104)
2026-08-11T06:28:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2113 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:46 dist-test-slave-07c2 krb5kdc[31688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:46 dist-test-slave-07c2 krb5kdc[31688](info): set up 2 sockets
Aug 11 06:28:46 dist-test-slave-07c2 krb5kdc[31688](info): commencing operation
krb5kdc: starting...
W20260811 06:28:47.517827 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:47 dist-test-slave-07c2 krb5kdc[31688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:47.822865 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33289
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:33289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:48.154075 31704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:48.154389 31704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:48.154542 31704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:48.161422 31704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:48.161559 31704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:48.161659 31704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:48.161785 31704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:48.161865 31704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:48.171170 31704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:33289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:33289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:48.172365 31704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:48.173630 31704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:48.185016 31709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:48.184975 31712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:48.184957 31710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:48.186074 31704 server_base.cc:1034] running on GCE node
I20260811 06:28:48.186796 31704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:48.188104 31704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:48.189328 31704 hybrid_clock.cc:648] HybridClock initialized: now 1786429728189303 us; error 40 us; skew 500 ppm
I20260811 06:28:48.189663 31704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:48 dist-test-slave-07c2 krb5kdc[31688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429728, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:48.197095 31704 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:48.198444 31704 webserver.cc:460] Webserver started at http://127.7.4.190:37609/ using document root <none> and password file <none>
I20260811 06:28:48.199103 31704 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:48.199237 31704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:48.199579 31704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:48.202427 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb"
format_stamp: "Formatted at 2026-08-11 06:28:48 on dist-test-slave-07c2"
server_key: "4cb4453e1d31e0ee8955ec2ab7f84b7e"
server_key_iv: "f6eaddaf488f19941fd9d7c1415f5712"
server_key_version: "encryptionkey@0"
I20260811 06:28:48.203115 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb"
format_stamp: "Formatted at 2026-08-11 06:28:48 on dist-test-slave-07c2"
server_key: "4cb4453e1d31e0ee8955ec2ab7f84b7e"
server_key_iv: "f6eaddaf488f19941fd9d7c1415f5712"
server_key_version: "encryptionkey@0"
I20260811 06:28:48.208138 31704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:28:48.211459 31722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:48.212666 31704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:48.212854 31704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb"
format_stamp: "Formatted at 2026-08-11 06:28:48 on dist-test-slave-07c2"
server_key: "4cb4453e1d31e0ee8955ec2ab7f84b7e"
server_key_iv: "f6eaddaf488f19941fd9d7c1415f5712"
server_key_version: "encryptionkey@0"
I20260811 06:28:48.213042 31704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:48.228842 31704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:48.232600 31704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:48.232875 31704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:48.250671 31704 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:33289
I20260811 06:28:48.250782 31780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:33289 every 8 connection(s)
I20260811 06:28:48.251983 31704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:48.255677 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:48.261245 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31704
I20260811 06:28:48.261452 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:48.262013 7186 external_mini_cluster.cc:1428] Setting key 669e6f14371bcac4a37fc6009dd26154
I20260811 06:28:48.266345 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb: Bootstrap starting.
I20260811 06:28:48.269778 31781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:48.271008 31781 log.cc:824] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:48.274087 31781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb: No bootstrap required, opened a new log
I20260811 06:28:48.275502 31784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:48.263787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60321 (local address 127.7.4.190:33289)
0811 06:28:48.264693 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:48.264704 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:48.264743 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:28:48.266351 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:48.266371 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:48.266750 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:48.267043 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:48.267687 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:48.269021 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:48.270940 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:48.271907 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:48.272118 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:48.273197 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:48.273204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:48.273216 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:48.273738 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:48.273811 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:48.273835 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:48.273947 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:48.274336 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 06:28:48.274517 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":474,"threads_started":1}
I20260811 06:28:48.279191 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:48.283197 31781 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } }
I20260811 06:28:48.283684 31781 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:48.283799 31781 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34f1d4cb4f2408283e8a4ebe52c9edb, State: Initialized, Role: FOLLOWER
I20260811 06:28:48.284396 31781 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } }
I20260811 06:28:48.284664 31781 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:48.284869 31781 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:48.285054 31781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:48.287988 31781 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } }
I20260811 06:28:48.288501 31781 leader_election.cc:302] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34f1d4cb4f2408283e8a4ebe52c9edb; no voters:
I20260811 06:28:48.289297 31781 leader_election.cc:288] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:48.289397 31791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:48.290702 31791 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [term 1 LEADER]: Becoming Leader. State: Replica: c34f1d4cb4f2408283e8a4ebe52c9edb, State: Running, Role: LEADER
I20260811 06:28:48.291339 31791 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } }
I20260811 06:28:48.291747 31784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:48.284550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53823 (local address 127.7.4.190:33289)
0811 06:28:48.284742 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:48.284749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:48.285166 (+ 417us) server_negotiation.cc:407] Connection header received
0811 06:28:48.285366 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:48.285370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:48.285422 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:48.285516 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:48.286119 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:48.286858 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:48.288727 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:48.289186 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:48.289345 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:48.290636 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:48.290645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:48.290651 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:48.290679 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:48.290738 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:48.290768 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:48.290905 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:48.291365 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 06:28:48.291419 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:28:48.292068 31781 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:48.293330 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } } }
I20260811 06:28:48.293854 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:48.294503 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34f1d4cb4f2408283e8a4ebe52c9edb. Latest consensus state: current_term: 1 leader_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34f1d4cb4f2408283e8a4ebe52c9edb" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 33289 } } }
I20260811 06:28:48.295944 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:48.300143 31798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:48.305586 31798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:48.315006 31798 catalog_manager.cc:1347] Generated new cluster ID: 2cd1c231b6094a378fa57ba83ae684ba
I20260811 06:28:48.315131 31798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:48.340809 31798 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:48.341977 31798 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:48.348914 31798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34f1d4cb4f2408283e8a4ebe52c9edb: Generated new TSK 0
I20260811 06:28:48.349664 31798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:48.355041 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31704
W20260811 06:28:48.356376 31776 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60321
W20260811 06:28:48.357111 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31704: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:48.357256 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31704
W20260811 06:28:48.376305 31686 connection.cc:535] client connection to 127.7.4.190:33289 recv error: Network error: failed to read from TLS socket (remote: 127.7.4.190:33289): Connection reset by peer (error 104)
2026-08-11T06:28:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:48 dist-test-slave-07c2 krb5kdc[31815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:48 dist-test-slave-07c2 krb5kdc[31815](info): set up 2 sockets
Aug 11 06:28:48 dist-test-slave-07c2 krb5kdc[31815](info): commencing operation
krb5kdc: starting...
W20260811 06:28:49.531227 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:49 dist-test-slave-07c2 krb5kdc[31815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:49.868077 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41013
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:41013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:50.204854 31829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:50.205178 31829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:50.205346 31829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:50.212291 31829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:50.212435 31829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:50.212545 31829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:50.212658 31829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:50.212759 31829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:50.222889 31829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:41013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:50.224207 31829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:50.225508 31829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:50.237329 31835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:50.237286 31834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:50.237365 31839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:50.238407 31829 server_base.cc:1034] running on GCE node
I20260811 06:28:50.239168 31829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:50.240588 31829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:50.241798 31829 hybrid_clock.cc:648] HybridClock initialized: now 1786429730241747 us; error 38 us; skew 500 ppm
I20260811 06:28:50.242141 31829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:50 dist-test-slave-07c2 krb5kdc[31815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:50.245334 31829 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:50.246812 31829 webserver.cc:460] Webserver started at http://127.7.4.190:38291/ using document root <none> and password file <none>
I20260811 06:28:50.247614 31829 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:50.247761 31829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:50.248092 31829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:50.251013 31829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "60a279a9d12745d88fbbda41f93266bc"
format_stamp: "Formatted at 2026-08-11 06:28:50 on dist-test-slave-07c2"
server_key: "33fc9b3870ae81f3c1ccf16204d180bf"
server_key_iv: "94509b05233a2bfb21c8e3ed29ff1190"
server_key_version: "encryptionkey@0"
I20260811 06:28:50.251878 31829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "60a279a9d12745d88fbbda41f93266bc"
format_stamp: "Formatted at 2026-08-11 06:28:50 on dist-test-slave-07c2"
server_key: "33fc9b3870ae81f3c1ccf16204d180bf"
server_key_iv: "94509b05233a2bfb21c8e3ed29ff1190"
server_key_version: "encryptionkey@0"
I20260811 06:28:50.256947 31829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:28:50.260421 31847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:50.261680 31829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:28:50.261873 31829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "60a279a9d12745d88fbbda41f93266bc"
format_stamp: "Formatted at 2026-08-11 06:28:50 on dist-test-slave-07c2"
server_key: "33fc9b3870ae81f3c1ccf16204d180bf"
server_key_iv: "94509b05233a2bfb21c8e3ed29ff1190"
server_key_version: "encryptionkey@0"
I20260811 06:28:50.262074 31829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:50.267701 31829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:50.271476 31829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:50.271765 31829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:50.290891 31829 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:41013
I20260811 06:28:50.290975 31903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:41013 every 8 connection(s)
I20260811 06:28:50.292316 31829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:50.295943 31904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:50.296190 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31829
I20260811 06:28:50.296357 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:50.296825 7186 external_mini_cluster.cc:1428] Setting key 19d6b1125a84abd9ebe6db482efbaa95
I20260811 06:28:50.304818 31907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:50.298515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50579 (local address 127.7.4.190:41013)
0811 06:28:50.299460 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:50.299474 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:50.299520 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:28:50.301137 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:50.301178 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:50.301617 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:50.302045 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:50.302970 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:50.302978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:50.303000 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:50.303021 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:50.303085 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:50.303120 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:50.303364 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:50.303502 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 06:28:50.303678 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":357,"threads_started":1}
I20260811 06:28:50.308357 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:50.310349 31904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc: Bootstrap starting.
I20260811 06:28:50.313495 31904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:50.314631 31904 log.cc:824] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:50.315562 31907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:50.313058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50039 (local address 127.7.4.190:41013)
0811 06:28:50.313223 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:50.313229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:50.313584 (+ 355us) server_negotiation.cc:407] Connection header received
0811 06:28:50.313749 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:50.313753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:50.313812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:50.313939 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:50.314799 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:50.314804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:50.314809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:50.314825 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:50.314867 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:50.314885 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:50.315012 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:50.315155 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:28:50.315224 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:28:50.317634 31904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc: No bootstrap required, opened a new log
I20260811 06:28:50.324707 31904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } }
I20260811 06:28:50.325030 31904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:50.325093 31904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a279a9d12745d88fbbda41f93266bc, State: Initialized, Role: FOLLOWER
I20260811 06:28:50.325601 31904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } }
I20260811 06:28:50.325810 31904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:50.325915 31904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:50.326026 31904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:50.328613 31904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } }
I20260811 06:28:50.329059 31904 leader_election.cc:302] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60a279a9d12745d88fbbda41f93266bc; no voters:
I20260811 06:28:50.329561 31904 leader_election.cc:288] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:50.329715 31915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:50.330451 31915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [term 1 LEADER]: Becoming Leader. State: Replica: 60a279a9d12745d88fbbda41f93266bc, State: Running, Role: LEADER
I20260811 06:28:50.331024 31915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } }
I20260811 06:28:50.331761 31904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:50.334209 31917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60a279a9d12745d88fbbda41f93266bc. Latest consensus state: current_term: 1 leader_uuid: "60a279a9d12745d88fbbda41f93266bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } } }
I20260811 06:28:50.334182 31916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60a279a9d12745d88fbbda41f93266bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a279a9d12745d88fbbda41f93266bc" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41013 } } }
I20260811 06:28:50.334676 31917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:50.334676 31916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:50.339250 31924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:50.344548 31924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:50.353385 31924 catalog_manager.cc:1347] Generated new cluster ID: e600dc1426bf4d86bb69e37802c2574b
I20260811 06:28:50.353523 31924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:50.378535 31924 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:50.379004 31924 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:50.395061 31924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60a279a9d12745d88fbbda41f93266bc: Generated new TSK 0
I20260811 06:28:50.395882 31924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:50.408239 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31829
W20260811 06:28:50.409317 31899 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50579
W20260811 06:28:50.410018 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31829: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:50.410198 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31829
2026-08-11T06:28:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2054 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:50 dist-test-slave-07c2 krb5kdc[31937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:50 dist-test-slave-07c2 krb5kdc[31937](info): set up 2 sockets
Aug 11 06:28:50 dist-test-slave-07c2 krb5kdc[31937](info): commencing operation
krb5kdc: starting...
W20260811 06:28:51.596406 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.007s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:51 dist-test-slave-07c2 krb5kdc[31937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:51.918154 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:43415
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:43415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:52.264833 31951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:52.265110 31951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:52.265214 31951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:52.272647 31951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:52.272751 31951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:52.272805 31951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:52.272866 31951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:52.272908 31951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:52.282197 31951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:43415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:43415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:52.283308 31951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:52.284518 31951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:52.295991 31956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:52.296042 31957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:52.295958 31959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:52.296888 31951 server_base.cc:1034] running on GCE node
I20260811 06:28:52.297636 31951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:52.298928 31951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:52.300127 31951 hybrid_clock.cc:648] HybridClock initialized: now 1786429732300082 us; error 43 us; skew 500 ppm
I20260811 06:28:52.300488 31951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:52 dist-test-slave-07c2 krb5kdc[31937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429732, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:52.303520 31951 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:52.304945 31951 webserver.cc:460] Webserver started at http://127.7.4.190:32995/ using document root <none> and password file <none>
I20260811 06:28:52.305588 31951 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:52.305743 31951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:52.306043 31951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:52.308985 31951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "e41977481b3b415ea9c61d1117ae3c68"
format_stamp: "Formatted at 2026-08-11 06:28:52 on dist-test-slave-07c2"
server_key: "64017b4c44874ec852ee660679f23c37"
server_key_iv: "74d8b35fd0ce0a6648d2b036b64fa45e"
server_key_version: "encryptionkey@0"
I20260811 06:28:52.309744 31951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "e41977481b3b415ea9c61d1117ae3c68"
format_stamp: "Formatted at 2026-08-11 06:28:52 on dist-test-slave-07c2"
server_key: "64017b4c44874ec852ee660679f23c37"
server_key_iv: "74d8b35fd0ce0a6648d2b036b64fa45e"
server_key_version: "encryptionkey@0"
I20260811 06:28:52.314581 31951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:28:52.317919 31966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:52.319191 31951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:28:52.319416 31951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "e41977481b3b415ea9c61d1117ae3c68"
format_stamp: "Formatted at 2026-08-11 06:28:52 on dist-test-slave-07c2"
server_key: "64017b4c44874ec852ee660679f23c37"
server_key_iv: "74d8b35fd0ce0a6648d2b036b64fa45e"
server_key_version: "encryptionkey@0"
I20260811 06:28:52.319614 31951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:52.349128 31951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:52.353125 31951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:52.353433 31951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:52.372191 31951 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:43415
I20260811 06:28:52.372232 32026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:43415 every 8 connection(s)
I20260811 06:28:52.373561 31951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:52.377266 32028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:52.379557 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 31951
I20260811 06:28:52.379745 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:52.380286 7186 external_mini_cluster.cc:1428] Setting key 4e2b51666ead64e278c44c2c53d8161d
I20260811 06:28:52.389883 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68: Bootstrap starting.
I20260811 06:28:52.393677 32028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:52.394981 32028 log.cc:824] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:28:52 dist-test-slave-07c2 krb5kdc[31937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:52.397907 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68: No bootstrap required, opened a new log
I20260811 06:28:52.404155 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:52.382097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33011 (local address 127.7.4.190:43415)
0811 06:28:52.382937 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:52.382949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:52.382992 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:28:52.384707 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:52.384728 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:52.385081 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:52.385388 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:52.387236 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:52.388734 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:52.390606 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:52.391242 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:52.391462 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:52.396757 (+ 5295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:52.396765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:52.396778 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:52.396812 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:52.400086 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:52.400800 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:52.400806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:52.400811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:52.400986 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:52.401393 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:52.401400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:52.401403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:52.401915 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:52.402260 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:52.402706 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 06:28:52.403116 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":408,"threads_started":1}
I20260811 06:28:52.405931 32028 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } }
I20260811 06:28:52.406354 32028 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:52.406510 32028 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41977481b3b415ea9c61d1117ae3c68, State: Initialized, Role: FOLLOWER
I20260811 06:28:52.407142 32028 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } }
I20260811 06:28:52.407446 32028 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:52.407598 32028 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:52.407780 32028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:52.408043 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:52.410878 32028 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } }
I20260811 06:28:52.411520 32028 leader_election.cc:302] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41977481b3b415ea9c61d1117ae3c68; no voters:
I20260811 06:28:52.412436 32028 leader_election.cc:288] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:52.412545 32037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:52.421188 32028 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:52.424948 32037 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [term 1 LEADER]: Becoming Leader. State: Replica: e41977481b3b415ea9c61d1117ae3c68, State: Running, Role: LEADER
I20260811 06:28:52.425660 32037 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } }
I20260811 06:28:52.429075 32040 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41977481b3b415ea9c61d1117ae3c68. Latest consensus state: current_term: 1 leader_uuid: "e41977481b3b415ea9c61d1117ae3c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } } }
I20260811 06:28:52.429473 32040 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:52.430305 32039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41977481b3b415ea9c61d1117ae3c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41977481b3b415ea9c61d1117ae3c68" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 43415 } } }
I20260811 06:28:52.430668 32039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:52.434881 32048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:52.440027 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:52.415998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52489 (local address 127.7.4.190:43415)
0811 06:28:52.416206 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:52.416215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:52.417357 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 06:28:52.417551 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:52.417560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:52.417658 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:52.417783 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:52.422597 (+ 4814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:52.424419 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:52.426421 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:52.426907 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:52.427030 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:52.430933 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:52.430943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:52.430951 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:52.431003 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:52.435660 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:52.436668 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:52.436677 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:52.436683 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:52.436867 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:52.437491 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:52.437499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:52.437505 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:52.438061 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:52.438379 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:52.439156 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 06:28:52.439491 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:28:52.440375 32048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:52.450181 32048 catalog_manager.cc:1347] Generated new cluster ID: 236a11e59a4a4a5aac4a9c0d31039be4
I20260811 06:28:52.450297 32048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:52.464620 32048 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:52.465848 32048 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:52.474449 32048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41977481b3b415ea9c61d1117ae3c68: Generated new TSK 0
I20260811 06:28:52.475199 32048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:52.484966 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 31951
I20260811 06:28:52.486042 32021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:28:52.578840 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 31951
2026-08-11T06:28:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2168 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:52 dist-test-slave-07c2 krb5kdc[32063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:52 dist-test-slave-07c2 krb5kdc[32063](info): set up 2 sockets
Aug 11 06:28:52 dist-test-slave-07c2 krb5kdc[32063](info): commencing operation
krb5kdc: starting...
W20260811 06:28:53.777275 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:53 dist-test-slave-07c2 krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:54.108209 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41169
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:41169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:54.446424 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:54.446720 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:54.446812 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:54.454053 32077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:54.454176 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:54.454272 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:54.454377 32077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:54.454438 32077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:54.464377 32077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:41169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:41169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:54.465622 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:54.466894 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:54.478777 32083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:54.478837 32087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:54.478875 32084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:54.479982 32077 server_base.cc:1034] running on GCE node
I20260811 06:28:54.480682 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:54.481914 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:54.483141 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786429734483105 us; error 46 us; skew 500 ppm
I20260811 06:28:54.483522 32077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:54 dist-test-slave-07c2 krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429734, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:54.494029 32077 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:54.495620 32077 webserver.cc:460] Webserver started at http://127.7.4.190:38329/ using document root <none> and password file <none>
I20260811 06:28:54.496326 32077 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:54.496479 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:54.496793 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:54.499830 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "65761ac5154048b5b5f4d572fb21bc03"
format_stamp: "Formatted at 2026-08-11 06:28:54 on dist-test-slave-07c2"
server_key: "d34a9f78c649095ef26777d370cbfadb"
server_key_iv: "b5b540a42a9337b40d5297b0f520c48d"
server_key_version: "encryptionkey@0"
I20260811 06:28:54.500623 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "65761ac5154048b5b5f4d572fb21bc03"
format_stamp: "Formatted at 2026-08-11 06:28:54 on dist-test-slave-07c2"
server_key: "d34a9f78c649095ef26777d370cbfadb"
server_key_iv: "b5b540a42a9337b40d5297b0f520c48d"
server_key_version: "encryptionkey@0"
I20260811 06:28:54.505898 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:28:54.510176 32094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:54.511551 32077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:54.511720 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "65761ac5154048b5b5f4d572fb21bc03"
format_stamp: "Formatted at 2026-08-11 06:28:54 on dist-test-slave-07c2"
server_key: "d34a9f78c649095ef26777d370cbfadb"
server_key_iv: "b5b540a42a9337b40d5297b0f520c48d"
server_key_version: "encryptionkey@0"
I20260811 06:28:54.511919 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:54.550258 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:54.555356 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:54.555685 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:54.576501 32077 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:41169
I20260811 06:28:54.576534 32155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:41169 every 8 connection(s)
I20260811 06:28:54.578027 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:54.580578 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 32077
I20260811 06:28:54.580807 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:54.581298 7186 external_mini_cluster.cc:1428] Setting key f960b552ec632374d84d5df95ae1d0f1
I20260811 06:28:54.582226 32156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:54.596282 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:54.583408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51887 (local address 127.7.4.190:41169)
0811 06:28:54.584268 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:54.584285 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:54.584335 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:28:54.586018 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:54.586045 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:54.586565 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:54.587027 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:54.587588 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:54.589664 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:54.591021 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:54.592064 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:54.592255 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:54.593426 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:54.593443 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:54.593460 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:54.594209 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:54.594311 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:54.594350 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:54.594476 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:54.594753 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:28:54.595023 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":357,"threads_started":1}
I20260811 06:28:54.597545 32156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03: Bootstrap starting.
I20260811 06:28:54.600358 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:54.601131 32156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:54.602084 32156 log.cc:824] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:54.605360 32156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03: No bootstrap required, opened a new log
I20260811 06:28:54.610960 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:54.604828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35183 (local address 127.7.4.190:41169)
0811 06:28:54.604999 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:54.605008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:54.605431 (+ 423us) server_negotiation.cc:407] Connection header received
0811 06:28:54.605651 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:54.605655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:54.605710 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:54.605843 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:54.606353 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:54.607121 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:54.608495 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:54.608958 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:54.609118 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:54.610195 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:54.610200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:54.610203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:54.610221 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:54.610271 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:54.610286 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:54.610409 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:54.610555 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:28:54.610643 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:28:54.613547 32156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } }
I20260811 06:28:54.613922 32156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:54.614029 32156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65761ac5154048b5b5f4d572fb21bc03, State: Initialized, Role: FOLLOWER
I20260811 06:28:54.614625 32156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } }
I20260811 06:28:54.614872 32156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:54.615025 32156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:54.615214 32156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:54.617677 32156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } }
I20260811 06:28:54.618141 32156 leader_election.cc:302] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65761ac5154048b5b5f4d572fb21bc03; no voters:
I20260811 06:28:54.618736 32156 leader_election.cc:288] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:54.618866 32167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:54.619642 32167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [term 1 LEADER]: Becoming Leader. State: Replica: 65761ac5154048b5b5f4d572fb21bc03, State: Running, Role: LEADER
I20260811 06:28:54.620203 32167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } }
I20260811 06:28:54.620855 32156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:54.623207 32168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65761ac5154048b5b5f4d572fb21bc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } } }
I20260811 06:28:54.623684 32168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:54.624284 32169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65761ac5154048b5b5f4d572fb21bc03. Latest consensus state: current_term: 1 leader_uuid: "65761ac5154048b5b5f4d572fb21bc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65761ac5154048b5b5f4d572fb21bc03" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 41169 } } }
I20260811 06:28:54.624629 32169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:54.625538 32173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:54.630105 32173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:54.638965 32173 catalog_manager.cc:1347] Generated new cluster ID: 2025569e800b4f1cbbe8ab6eebd430bc
I20260811 06:28:54.639461 32173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:54.653283 32173 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:54.654973 32173 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:54.677232 32173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65761ac5154048b5b5f4d572fb21bc03: Generated new TSK 0
I20260811 06:28:54.678166 32173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:54.686741 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32077
W20260811 06:28:54.687764 32149 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51887
W20260811 06:28:54.688484 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32077: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:54.688642 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 32077
2026-08-11T06:28:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:54 dist-test-slave-07c2 krb5kdc[32190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:54 dist-test-slave-07c2 krb5kdc[32190](info): set up 2 sockets
Aug 11 06:28:54 dist-test-slave-07c2 krb5kdc[32190](info): commencing operation
krb5kdc: starting...
W20260811 06:28:55.876505 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:56 dist-test-slave-07c2 krb5kdc[32190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:56.202289 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36507
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:35163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:36507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:56.539477 32205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:56.539767 32205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:56.539867 32205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:56.547093 32205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:56.547224 32205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:56.547272 32205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:56.547406 32205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:56.547499 32205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:56.557657 32205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:35163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:36507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:56.559227 32205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:56.560571 32205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:56.572417 32211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:56.572607 32214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:56.572899 32212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:56.573392 32205 server_base.cc:1034] running on GCE node
I20260811 06:28:56.574128 32205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:56.575546 32205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:56.576781 32205 hybrid_clock.cc:648] HybridClock initialized: now 1786429736576748 us; error 42 us; skew 500 ppm
I20260811 06:28:56.577116 32205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:56 dist-test-slave-07c2 krb5kdc[32190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429736, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:56.580327 32205 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:56.581868 32205 webserver.cc:460] Webserver started at http://127.7.4.190:36157/ using document root <none> and password file <none>
I20260811 06:28:56.582603 32205 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:56.582800 32205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:56.583087 32205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:56.585976 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "d38fb961628c47d4970d2ec491a24b97"
format_stamp: "Formatted at 2026-08-11 06:28:56 on dist-test-slave-07c2"
server_key: "69b8fbd50767a5eae7932ae61e66ad7d"
server_key_iv: "3e4d57e8c8a4c645d34b7766989107ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:56.586668 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "d38fb961628c47d4970d2ec491a24b97"
format_stamp: "Formatted at 2026-08-11 06:28:56 on dist-test-slave-07c2"
server_key: "69b8fbd50767a5eae7932ae61e66ad7d"
server_key_iv: "3e4d57e8c8a4c645d34b7766989107ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:56.591717 32205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:28:56.594951 32221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:56.596333 32205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:28:56.596531 32205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "d38fb961628c47d4970d2ec491a24b97"
format_stamp: "Formatted at 2026-08-11 06:28:56 on dist-test-slave-07c2"
server_key: "69b8fbd50767a5eae7932ae61e66ad7d"
server_key_iv: "3e4d57e8c8a4c645d34b7766989107ec"
server_key_version: "encryptionkey@0"
I20260811 06:28:56.596722 32205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:56.602598 32205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:56.606487 32205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:56.606773 32205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:56.625285 32205 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:36507
I20260811 06:28:56.625330 32279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:36507 every 8 connection(s)
I20260811 06:28:56.626636 32205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:56.630267 32280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:56.633827 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 32205
I20260811 06:28:56.634018 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:56.634455 7186 external_mini_cluster.cc:1428] Setting key 4392d1ff2d4d8fc0cdb900cc344c8757
I20260811 06:28:56.641922 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:56.636265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44451 (local address 127.7.4.190:36507)
0811 06:28:56.637030 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:56.637041 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:56.637081 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:28:56.638638 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:56.638659 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:56.639008 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:56.639390 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:56.640391 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:56.640400 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:56.640412 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:56.640435 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:56.640486 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:56.640506 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:56.640698 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:56.640835 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:28:56.641024 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":362,"threads_started":1}
I20260811 06:28:56.642763 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97: Bootstrap starting.
I20260811 06:28:56.645484 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:56.646080 32280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:56.647053 32280 log.cc:824] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:56.650463 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97: No bootstrap required, opened a new log
I20260811 06:28:56.652639 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:56.650050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40317 (local address 127.7.4.190:36507)
0811 06:28:56.650212 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:56.650218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:56.650648 (+ 430us) server_negotiation.cc:407] Connection header received
0811 06:28:56.650840 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:56.650847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:56.650895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:56.651004 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:56.651963 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:56.651968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:56.651975 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:28:56.652001 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:56.652040 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:28:56.652054 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:56.652210 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:56.652313 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 06:28:56.652380 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:28:56.658069 32280 raft_consensus.cc:348] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } }
I20260811 06:28:56.658488 32280 raft_consensus.cc:374] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:28:56.658604 32280 raft_consensus.cc:729] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38fb961628c47d4970d2ec491a24b97, State: Initialized, Role: FOLLOWER
I20260811 06:28:56.659173 32280 consensus_queue.cc:260] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } }
I20260811 06:28:56.659511 32280 raft_consensus.cc:388] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:28:56.659668 32280 raft_consensus.cc:482] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:28:56.659893 32280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:56.662127 32280 raft_consensus.cc:504] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } }
I20260811 06:28:56.662596 32280 leader_election.cc:302] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d38fb961628c47d4970d2ec491a24b97; no voters:
I20260811 06:28:56.663127 32280 leader_election.cc:288] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:28:56.663257 32290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:56.664039 32290 raft_consensus.cc:686] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [term 1 LEADER]: Becoming Leader. State: Replica: d38fb961628c47d4970d2ec491a24b97, State: Running, Role: LEADER
I20260811 06:28:56.664662 32290 consensus_queue.cc:237] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } }
I20260811 06:28:56.665481 32280 sys_catalog.cc:564] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:56.667701 32294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d38fb961628c47d4970d2ec491a24b97. Latest consensus state: current_term: 1 leader_uuid: "d38fb961628c47d4970d2ec491a24b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } } }
I20260811 06:28:56.668032 32294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:56.668476 32293 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d38fb961628c47d4970d2ec491a24b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38fb961628c47d4970d2ec491a24b97" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 36507 } } }
I20260811 06:28:56.668816 32293 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:56.672585 32299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:56.677834 32299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:28:56.686496 32299 catalog_manager.cc:1347] Generated new cluster ID: 9075f68548d746378b7e1da05ab79a70
I20260811 06:28:56.686604 32299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:28:56.719257 32299 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:28:56.719792 32299 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:28:56.728011 32299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d38fb961628c47d4970d2ec491a24b97: Generated new TSK 0
I20260811 06:28:56.728760 32299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:28:56.733865 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32205
W20260811 06:28:56.734953 32274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44451
W20260811 06:28:56.735677 7186 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32205: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:28:56.735832 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 32205
2026-08-11T06:28:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2051 ms)
[----------] 18 tests from AuthTokenIssuingTest (37585 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:28:56 dist-test-slave-07c2 krb5kdc[32315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:28:56 dist-test-slave-07c2 krb5kdc[32315](info): set up 2 sockets
Aug 11 06:28:56 dist-test-slave-07c2 krb5kdc[32315](info): commencing operation
krb5kdc: starting...
W20260811 06:28:57.932240 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:28:58 dist-test-slave-07c2 krb5kdc[32315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:28:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:28:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:28:58.265036 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34127
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:28:58.609980 32330 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:28:58.610261 32330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:58.610347 32330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:58.610495 32330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:58.617548 32330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:58.617620 32330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:58.617683 32330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:58.617748 32330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:58.617874 32330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:58.627317 32330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:34127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:58.628454 32330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:58.629772 32330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:58.641247 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:58.641297 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:58.641202 32339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:58.642292 32330 server_base.cc:1034] running on GCE node
I20260811 06:28:58.643074 32330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:58.644418 32330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:58.645663 32330 hybrid_clock.cc:648] HybridClock initialized: now 1786429738645603 us; error 37 us; skew 500 ppm
I20260811 06:28:58.645984 32330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:58 dist-test-slave-07c2 krb5kdc[32315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:58.649096 32330 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:28:58.650484 32330 webserver.cc:460] Webserver started at http://127.7.4.190:40175/ using document root <none> and password file <none>
I20260811 06:28:58.651168 32330 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:58.651337 32330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:58.651669 32330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:58.655052 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "807d1b417c364257b720879ab6f8d1b6"
format_stamp: "Formatted at 2026-08-11 06:28:58 on dist-test-slave-07c2"
server_key: "fb99f8f12884e32f98b5d6c6372cff73"
server_key_iv: "08564b01507743b4e4829054137d52b1"
server_key_version: "encryptionkey@0"
I20260811 06:28:58.655890 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "807d1b417c364257b720879ab6f8d1b6"
format_stamp: "Formatted at 2026-08-11 06:28:58 on dist-test-slave-07c2"
server_key: "fb99f8f12884e32f98b5d6c6372cff73"
server_key_iv: "08564b01507743b4e4829054137d52b1"
server_key_version: "encryptionkey@0"
I20260811 06:28:58.660710 32330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 06:28:58.664104 32347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:58.665361 32330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:28:58.665532 32330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "807d1b417c364257b720879ab6f8d1b6"
format_stamp: "Formatted at 2026-08-11 06:28:58 on dist-test-slave-07c2"
server_key: "fb99f8f12884e32f98b5d6c6372cff73"
server_key_iv: "08564b01507743b4e4829054137d52b1"
server_key_version: "encryptionkey@0"
I20260811 06:28:58.665756 32330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:58.697831 32330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:58.701746 32330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:58.702020 32330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:58.721002 32330 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:34127
I20260811 06:28:58.721038 32403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:34127 every 8 connection(s)
I20260811 06:28:58.722334 32330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:28:58.725944 32404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:58.728971 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 32330
I20260811 06:28:58.729197 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:28:58.729861 7186 external_mini_cluster.cc:1428] Setting key d1b3d2db02aec905b29ffcec1d06d559
I20260811 06:28:58.732527 32404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:28:58 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:58.755537 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.735549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34127 (local address 127.0.0.1:34880)
0811 06:28:58.736674 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.736712 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:58.738373 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:58.739161 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:58.739168 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:58.739863 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:58.740187 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:58.740204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:58.741743 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:58.741749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:58.742799 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:58.742811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:58.743794 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:58.743801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:58.744001 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:58.744655 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:58.744676 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:58.747265 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:58.751761 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:58.751772 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:58.751779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:58.752164 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:58.752669 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:58.752673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:58.752675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:58.752817 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:58.753775 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:58.753780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:58.753964 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 06:28:58.754155 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 06:28:58.754510 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":3,"thread_start_us":734,"threads_started":1}
I20260811 06:28:58.755575 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.735548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.7.4.190:34127)
0811 06:28:58.736380 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:58.736396 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:58.736825 (+ 429us) server_negotiation.cc:407] Connection header received
0811 06:28:58.738708 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:58.738713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:58.738939 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:58.739137 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:58.740407 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:58.741583 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:58.742940 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:58.743559 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:58.743756 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:58.747586 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:58.747599 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:58.747615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:58.747658 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:58.751634 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:58.752315 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:58.752325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:58.752331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:58.752548 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:58.752952 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:58.752959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:58.752965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:58.753639 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:58.753837 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:58.754382 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 06:28:58.754684 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":391,"threads_started":1}
I20260811 06:28:58.758287 32404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:28:58.759343 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.758836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43126)
0811 06:28:58.759025 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.759160 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:28:58.759591 32349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
I20260811 06:28:58.761266 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.760765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43134)
0811 06:28:58.760952 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.761051 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:28:58.761495 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:38111: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.4.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
I20260811 06:28:58.803838 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:38111
--webserver_interface=127.7.4.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:28:58.810931 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } attempt: 1
I20260811 06:28:58.812161 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.811722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43140)
0811 06:28:58.811916 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.812043 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:28:58.813128 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.812757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43152)
0811 06:28:58.812932 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.813006 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 06:28:58.813321 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:38111: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
I20260811 06:28:58.881573 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } attempt: 2
I20260811 06:28:58.882784 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.882289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43158)
0811 06:28:58.882555 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.882645 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:28:58.883798 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.883444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43160)
0811 06:28:58.883587 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.883675 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:28:58.883994 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:38111: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
I20260811 06:28:58.975344 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } attempt: 3
I20260811 06:28:58.976470 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.976037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43170)
0811 06:28:58.976229 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.976330 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:28:58.977445 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:58.977095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43186)
0811 06:28:58.977248 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:58.977317 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:28:58.977626 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:38111: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
I20260811 06:28:59.120875 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } attempt: 4
I20260811 06:28:59.122097 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.121550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43192)
0811 06:28:59.121882 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.121967 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 06:28:59.123085 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.122730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43204)
0811 06:28:59.122884 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.122969 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:28:59.123302 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:38111: Network error: Client connection negotiation failed: client connection to 127.7.4.189:38111: connect: Connection refused (error 111)
W20260811 06:28:59.141674 32411 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:28:59.142012 32411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:59.142138 32411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:59.142277 32411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:59.149427 32411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:59.149541 32411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:59.149675 32411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:59.149763 32411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:59.149876 32411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:59.159516 32411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:38111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.4.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:59.160753 32411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:59.162038 32411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:59.173993 32416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:59.173949 32419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:59.174031 32417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:59.175002 32411 server_base.cc:1034] running on GCE node
I20260811 06:28:59.175787 32411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:59.177003 32411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:59.178238 32411 hybrid_clock.cc:648] HybridClock initialized: now 1786429739178168 us; error 29 us; skew 500 ppm
I20260811 06:28:59.178515 32411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:59.181605 32411 init.cc:377] Logged in from keytab as kudu/127.7.4.189@KRBTEST.COM (short username kudu)
I20260811 06:28:59.183020 32411 webserver.cc:460] Webserver started at http://127.7.4.189:45801/ using document root <none> and password file <none>
I20260811 06:28:59.183753 32411 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:59.183872 32411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:59.184212 32411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:59.187552 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data/instance:
uuid: "a845f715b84e42c887d09e3df234ce12"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "0de0b3f062ae38344c90de23c4a06a86"
server_key_iv: "cd4d885bdaad8fa7cbf26fa42720b376"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.188328 32411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal/instance:
uuid: "a845f715b84e42c887d09e3df234ce12"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "0de0b3f062ae38344c90de23c4a06a86"
server_key_iv: "cd4d885bdaad8fa7cbf26fa42720b376"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.193210 32411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:28:59.196417 32428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:59.197618 32411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:28:59.197809 32411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "a845f715b84e42c887d09e3df234ce12"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "0de0b3f062ae38344c90de23c4a06a86"
server_key_iv: "cd4d885bdaad8fa7cbf26fa42720b376"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.198026 32411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:59.219127 32411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:59.222813 32411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:59.223068 32411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:59.242159 32411 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.189:38111
I20260811 06:28:59.242178 32484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.189:38111 every 8 connection(s)
I20260811 06:28:59.243517 32411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
I20260811 06:28:59.245121 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 32411
I20260811 06:28:59.245335 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-1/wal/instance
I20260811 06:28:59.245882 7186 external_mini_cluster.cc:1428] Setting key 27ca99da4884121e66baf409ee8a40ac
I20260811 06:28:59.247798 32485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:59.257547 32485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:59.281586 32488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.260698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.7.4.190:34127)
0811 06:28:59.261211 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.261219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.262289 (+ 1070us) server_negotiation.cc:407] Connection header received
0811 06:28:59.264525 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.264529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.264593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.264694 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.266796 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.267575 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.269990 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.270412 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.270559 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.274741 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.274749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.274757 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.274815 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.278538 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.279310 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.279318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.279326 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.279477 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.279863 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.279869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.279872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.280149 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.280300 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.281005 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.281222 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":249,"threads_started":1}
I20260811 06:28:59.282471 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.261238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34127 (local address 127.0.0.1:34892)
0811 06:28:59.262010 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.262054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.264251 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.264939 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.264951 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.265730 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.266563 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.266590 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.267742 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.267750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.269853 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.269870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.270641 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.270663 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.271048 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.271768 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.271807 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.274517 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.278676 (+ 4159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.278681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.278693 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.279166 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.279585 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.279588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.279591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.279745 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.280296 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.280304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.280610 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.280902 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.281374 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":354,"threads_started":1}
WARNING: no policy specified for kudu/127.7.4.188@KRBTEST.COM; defaulting to no policy
I20260811 06:28:59.284164 32485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.188@KRBTEST.COM" created.
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:28:59.299201 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.284796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43208)
0811 06:28:59.284987 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.285002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.285079 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.286121 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.286126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.286589 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.286736 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.286750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.288224 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.288233 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.289759 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.289788 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.290570 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.290578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.290799 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.291391 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.291418 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.293847 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.296828 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.296834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.296838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.297166 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.297517 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.297523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.297526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.297658 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.298250 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.298255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.298403 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.298553 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.298811 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 06:28:59.299304 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.285044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.7.4.189:38111)
0811 06:28:59.285634 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.285640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.285663 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:28:59.285701 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.285706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.285982 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.286126 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.286963 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.288042 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.289911 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.290419 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.290513 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.294049 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.294059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.294071 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.294103 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.296705 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.297299 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.297305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.297307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.297411 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.297758 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.297763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.297766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.298093 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.298336 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.298712 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.298911 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":317,"threads_started":1}
I20260811 06:28:59.301478 32485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:28:59.302718 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.302177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39604)
0811 06:28:59.302351 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.302550 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:28:59.303004 32429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.304761 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.304453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39606)
0811 06:28:59.304584 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.304664 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:28:59.305105 32485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
I20260811 06:28:59.326000 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:46095
--webserver_interface=127.7.4.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:28:59.354420 32485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 1
I20260811 06:28:59.355716 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.355216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39612)
0811 06:28:59.355455 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.355550 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 06:28:59.356778 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.356407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39620)
0811 06:28:59.356576 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.356670 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:28:59.356989 32485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.422559 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } attempt: 5
I20260811 06:28:59.425206 32485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 2
I20260811 06:28:59.426118 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.425691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39628)
0811 06:28:59.425854 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.425941 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:28:59.428164 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.427756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39644)
0811 06:28:59.427897 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.427976 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:28:59.428521 32485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:28:59.437374 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.423362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43214)
0811 06:28:59.423599 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.423622 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.423764 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.424154 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.424157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.424519 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.424711 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.424720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.426189 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.426195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.427092 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.427104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.428177 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.428186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.428415 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.429028 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.429046 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.431048 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.434927 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.434931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.434934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.435359 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.435788 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.435791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.435793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.435922 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.436531 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.436534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.436743 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.436950 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.437148 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 06:28:59.437559 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.423587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.7.4.189:38111)
0811 06:28:59.423801 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.423808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.423831 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:28:59.423896 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.423902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.423997 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.424112 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.424863 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.426000 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.427250 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.427990 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.428142 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.431240 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.431246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.431251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.431313 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.434791 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.435482 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.435488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.435493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.435673 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.436037 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.436044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.436049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.436419 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.436591 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.437070 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.437294 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:28:59.438601 32404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:28:59.439610 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.439226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39646)
0811 06:28:59.439416 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.439486 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 06:28:59.440513 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.440205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39652)
0811 06:28:59.440342 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.440416 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:28:59.440735 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.491976 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 1
I20260811 06:28:59.493333 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.492813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39664)
0811 06:28:59.493031 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.493140 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 06:28:59.494329 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.494028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39680)
0811 06:28:59.494161 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.494229 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:28:59.494586 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.519785 32485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 3
I20260811 06:28:59.521142 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.520453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39694)
0811 06:28:59.520756 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.520902 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 06:28:59.522437 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.521870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39710)
0811 06:28:59.522151 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.522261 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 06:28:59.522737 32485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.562842 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 2
I20260811 06:28:59.564152 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.563619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39722)
0811 06:28:59.563873 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.563967 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 06:28:59.565275 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.564956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39726)
0811 06:28:59.565090 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.565173 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:28:59.565515 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.666007 32485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 4
I20260811 06:28:59.667343 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.666735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39734)
0811 06:28:59.667004 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.667123 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:28:59.668452 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.668072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39738)
0811 06:28:59.668241 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.668326 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:28:59.668722 32485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
I20260811 06:28:59.671718 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 3
I20260811 06:28:59.672674 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.672277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39754)
0811 06:28:59.672433 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.672508 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:28:59.673681 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.673294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39756)
0811 06:28:59.673502 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.673580 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 06:28:59.673894 32404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:46095: Network error: Client connection negotiation failed: client connection to 127.7.4.188:46095: connect: Connection refused (error 111)
W20260811 06:28:59.677837 32495 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:28:59.678196 32495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:28:59.678320 32495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:28:59.678454 32495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:28:59.685738 32495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:28:59.685860 32495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:28:59.685968 32495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:28:59.686056 32495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:28:59.686163 32495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:28:59.696065 32495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:34127,127.7.4.189:38111,127.7.4.188:46095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:46095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.4.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:28:59.697358 32495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:28:59.698669 32495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:28:59.710754 32500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:59.710795 32501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:28:59.710832 32503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:28:59.712006 32495 server_base.cc:1034] running on GCE node
I20260811 06:28:59.712725 32495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:28:59.714007 32495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:28:59.715257 32495 hybrid_clock.cc:648] HybridClock initialized: now 1786429739715192 us; error 65 us; skew 500 ppm
I20260811 06:28:59.715628 32495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:28:59.724078 32495 init.cc:377] Logged in from keytab as kudu/127.7.4.188@KRBTEST.COM (short username kudu)
I20260811 06:28:59.725822 32495 webserver.cc:460] Webserver started at http://127.7.4.188:46599/ using document root <none> and password file <none>
I20260811 06:28:59.726640 32495 fs_manager.cc:359] Metadata directory not provided
I20260811 06:28:59.726769 32495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:28:59.727193 32495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:28:59.731240 32495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data/instance:
uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "11efff493ecb72b3e210b2b9aa8397a1"
server_key_iv: "d8f010a1a8173a7911e22e4df3c0cdb0"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.732057 32495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal/instance:
uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "11efff493ecb72b3e210b2b9aa8397a1"
server_key_iv: "d8f010a1a8173a7911e22e4df3c0cdb0"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.736931 32495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:28:59.740164 32510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:28:59.741415 32495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:28:59.741590 32495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e"
format_stamp: "Formatted at 2026-08-11 06:28:59 on dist-test-slave-07c2"
server_key: "11efff493ecb72b3e210b2b9aa8397a1"
server_key_iv: "d8f010a1a8173a7911e22e4df3c0cdb0"
server_key_version: "encryptionkey@0"
I20260811 06:28:59.741804 32495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:28:59.766559 32495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:28:59.770720 32495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:28:59.770990 32495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:28:59.789672 32495 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.188:46095
I20260811 06:28:59.789705 32566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.188:46095 every 8 connection(s)
I20260811 06:28:59.790951 32495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
I20260811 06:28:59.794417 32567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:28:59.798419 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 32495
I20260811 06:28:59.798597 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429373903770-7186-0/minicluster-data/master-2/wal/instance
I20260811 06:28:59.799152 7186 external_mini_cluster.cc:1428] Setting key 3bc5d56314e15899c83a989380a9bd8b
I20260811 06:28:59.801738 32567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:59.821167 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.805758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.7.4.190:34127)
0811 06:28:59.806349 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.806355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.806874 (+ 519us) server_negotiation.cc:407] Connection header received
0811 06:28:59.807053 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.807058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.807133 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.807264 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.808448 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.809220 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.810863 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.811461 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.811558 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.815514 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.815519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.815522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.815553 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.818077 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.818751 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.818755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.818757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.818884 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.819277 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.819282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.819304 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.819571 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.819700 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.820352 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.820574 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":270,"threads_started":1}
I20260811 06:28:59.822122 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.801131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.7.4.188:46095)
0811 06:28:59.801954 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.801964 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.802003 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:28:59.803849 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.803870 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.804260 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.804549 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.806419 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.808043 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.809534 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.810164 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.810300 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.815052 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.815063 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.815077 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.815130 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.818114 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.818723 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.818730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.818733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.818891 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.819351 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.819359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.819361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.819961 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.820270 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.820569 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.820827 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":593,"thread_start_us":388,"threads_started":1}
I20260811 06:28:59.822163 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.805733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:34127 (local address 127.0.0.1:34908)
0811 06:28:59.806733 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.806774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.806916 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.807377 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.807381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.807958 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.808295 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.808310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.809386 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.809392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.810646 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.810662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.811695 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.811705 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.811912 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.812525 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.812558 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.815322 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.818203 (+ 2881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.818215 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.818227 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.818628 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.818992 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.818997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.819001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.819164 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.819719 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.819726 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.819967 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.820249 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.820731 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":1,"thread_start_us":586,"threads_started":1}
I20260811 06:28:59.823882 32567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:28:59.837525 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.824520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:38111 (local address 127.0.0.1:43226)
0811 06:28:59.824735 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.824755 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.824846 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.825206 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.825211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.825662 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.825869 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.825884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.826843 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.826848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.827958 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.827973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.828704 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.828710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.828942 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.829536 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.829553 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.831818 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.835063 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.835069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.835073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.835518 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.835928 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.835933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.835936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.836099 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.836620 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.836624 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.836759 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.836890 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.837152 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 06:28:59.837559 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.824670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.7.4.189:38111)
0811 06:28:59.824881 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.824889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.824909 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:28:59.824971 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.824974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.825051 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.825155 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.826021 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.826666 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.828115 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.828545 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.828640 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.832006 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.832011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.832014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.832045 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.834915 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.835635 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.835642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.835645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.835798 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.836226 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.836230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.836232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.836487 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.836652 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.836994 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.837199 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:28:59.838490 32567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:28:59.839843 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.825163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.7.4.190:34127)
0811 06:28:59.825346 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.825350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.825433 (+ 83us) server_negotiation.cc:407] Connection header received
0811 06:28:59.825585 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.825588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.825645 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.825715 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.827026 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.827748 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.829249 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.829762 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.829856 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.834223 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.834227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.834230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.834258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.836862 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.837890 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.837895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.837897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.838052 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.838456 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.838460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.838462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.838751 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.839023 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.839304 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.839530 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:28:59.851109 32404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 4
I20260811 06:28:59.852808 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.839276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39788)
0811 06:28:59.839729 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.839746 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.839850 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.840265 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.840276 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.840569 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.840687 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.840694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.841826 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.841834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.843247 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.843262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.843940 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.843945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.844184 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.844690 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.844741 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.846879 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.850421 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.850428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.850431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.850764 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.851160 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.851165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.851168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.851372 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.852113 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.852118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.852270 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.852375 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.852571 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260811 06:28:59.853689 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.839581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.7.4.188:46095)
0811 06:28:59.839749 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.839753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.839792 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:28:59.840055 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.840061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.840117 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.840180 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.840824 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.841685 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.843409 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.843790 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.843920 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.847102 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.847108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.847114 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.847154 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.850293 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.850902 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.850910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.850913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.851050 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.851505 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.851513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.851517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.851889 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.852992 (+ 1103us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.853118 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.853382 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 06:28:59.857780 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.841434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.7.4.189:38111)
0811 06:28:59.841593 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.841599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.841649 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:28:59.841879 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.841882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.841936 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.842001 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.843554 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.844223 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.845436 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.846049 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.846190 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.851341 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.851347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.851352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.851388 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.854825 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.855691 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.855696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.855699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.855858 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.856340 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.856343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.856345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.856594 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.856884 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.857164 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.857386 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:28:59.860282 32567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: Bootstrap starting.
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429738, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:28:59.863212 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:28:59.864722 32567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:59.866156 32567 log.cc:824] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:59.868964 32407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.851720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39802)
0811 06:28:59.851969 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.851986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.852103 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.853901 (+ 1798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.853906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.854388 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.854528 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.854539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.856266 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.856279 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.857703 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.857719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.858647 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.858653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.858843 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.859453 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.859469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.861565 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.865764 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.865771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.865774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.866358 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.866921 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.866929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.866935 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.867104 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.867835 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.867841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.868057 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.868206 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.868561 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 06:28:59.865967 32377 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "807d1b417c364257b720879ab6f8d1b6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34922
I20260811 06:28:59.869931 32567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: No bootstrap required, opened a new log
I20260811 06:28:59.870183 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.851785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.7.4.188:46095)
0811 06:28:59.853577 (+ 1792us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.853582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.853603 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:28:59.853645 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.853650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.853722 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.854648 (+ 926us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.854863 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.856113 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.857880 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.858466 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.858611 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.861807 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.861815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.861819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.861856 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.865597 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.866546 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.866550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.866553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.866727 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.867228 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.867234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.867236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.867648 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.869419 (+ 1771us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.869548 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.869745 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668,"thread_start_us":1461,"threads_started":1}
I20260811 06:28:59.875499 32404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: Bootstrap starting.
I20260811 06:28:59.877045 32567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.877373 32567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc8e44f5baa47d3b04d867f6be2ed0e, State: Initialized, Role: FOLLOWER
I20260811 06:28:59.877846 32567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.878767 32576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:28:59.879141 32404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: Neither blocks nor log segments found. Creating new log.
I20260811 06:28:59.879144 32576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:28:59.879833 32567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:59.880451 32404 log.cc:824] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:28:59.883963 32404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: No bootstrap required, opened a new log
W20260811 06:28:59.889930 32590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:28:59.890084 32590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:28:59.891754 32404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.892167 32404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807d1b417c364257b720879ab6f8d1b6, State: Initialized, Role: FOLLOWER
I20260811 06:28:59.892683 32404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.893903 32592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:28:59.894650 32592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:28:59.895025 32404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:28:59.895129 32377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:28:59.895344 32377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:59.898706 32377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.887698 32495 master_runner.cc:186] Error getting master registration for 127.7.4.190:34127: OK, instance_id { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" instance_seqno: 1786429738715683 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:28:59.900418 32495 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:28:59.901715 32377 leader_election.cc:288] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [CANDIDATE]: Term 1 election: Requested vote from peers a845f715b84e42c887d09e3df234ce12 (127.7.4.189:38111), 0bc8e44f5baa47d3b04d867f6be2ed0e (127.7.4.188:46095)
I20260811 06:28:59.902644 32539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "807d1b417c364257b720879ab6f8d1b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e"
I20260811 06:28:59.903370 32539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:28:59.908010 32539 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807d1b417c364257b720879ab6f8d1b6 in term 1.
I20260811 06:28:59.908874 32348 leader_election.cc:302] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc8e44f5baa47d3b04d867f6be2ed0e, 807d1b417c364257b720879ab6f8d1b6; no voters:
I20260811 06:28:59.909538 32592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:28:59.910300 32592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [term 1 LEADER]: Becoming Leader. State: Replica: 807d1b417c364257b720879ab6f8d1b6, State: Running, Role: LEADER
I20260811 06:28:59.910912 32592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:28:59.923468 32598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807d1b417c364257b720879ab6f8d1b6. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:28:59.924158 32598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:28:59.928885 32608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:28:59.954675 32608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 06:28:59.954969 32353 tablet.cc:2369] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:28:59.968307 32485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } attempt: 5
Aug 11 06:28:59 dist-test-slave-07c2 krb5kdc[32315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429739, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:28:59.986070 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.936428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.7.4.188:46095)
0811 06:28:59.936660 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.936669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.945395 (+ 8726us) server_negotiation.cc:407] Connection header received
0811 06:28:59.945708 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.945716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.945831 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.945933 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.948265 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.949508 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.954944 (+ 5436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.955928 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.956056 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.966107 (+ 10051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.966114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.966136 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.966178 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.978086 (+ 11908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.979031 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.979038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.979045 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.979218 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.979767 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.979772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.979775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.980194 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.980468 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.981215 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.981442 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:28:59.989002 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.969097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:46095 (local address 127.0.0.1:39820)
0811 06:28:59.969468 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 06:28:59.969486 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:28:59.969594 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:28:59.970823 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:28:59.970828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:28:59.971357 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:28:59.971574 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.971586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.973000 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.973005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.976671 (+ 3666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:28:59.976687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.977960 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.977967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:28:59.978190 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.978779 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:28:59.978807 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:28:59.981717 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:28:59.985629 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.985634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.985638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.986578 (+ 940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.987021 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:28:59.987026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:28:59.987029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:28:59.987201 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:28:59.987967 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:28:59.987974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:28:59.988142 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 06:28:59.988281 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:28:59.988592 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260811 06:28:59.989702 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.934393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.7.4.190:34127)
0811 06:28:59.934598 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.934604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.944143 (+ 9539us) server_negotiation.cc:407] Connection header received
0811 06:28:59.944368 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.944374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.944459 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.944585 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.949948 (+ 5363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.951123 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.955156 (+ 4033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.955852 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.955955 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.973198 (+ 17243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.973205 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.973209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.973296 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.983102 (+ 9806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.984059 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.984079 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.984085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.984244 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.985201 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.985208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.985212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.985661 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.985996 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.989065 (+ 3069us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.989314 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:28:59.990288 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.969785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39820 (local address 127.7.4.188:46095)
0811 06:28:59.970217 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.970223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.970242 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:28:59.970285 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.970289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.970357 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.970459 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.971749 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.972827 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.976848 (+ 4021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.977562 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.977696 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.981927 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.981934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.981939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.981974 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.985456 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.986718 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.986722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.986725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.986876 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.987374 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.987379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.987381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.987754 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:28:59.987902 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:28:59.989466 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0811 06:28:59.989691 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260811 06:28:59.974498 32330 master_runner.cc:186] Error getting master registration for 127.7.4.189:38111: OK, instance_id { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" instance_seqno: 1786429739236897 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:28:59.991784 32330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:28:59.995268 32539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [term 1 FOLLOWER]: Refusing update from remote peer 807d1b417c364257b720879ab6f8d1b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:28:59.996347 32598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:28:59.996974 32485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: Bootstrap starting.
I20260811 06:29:00.001145 32485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:00.002197 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:28:59.948309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.7.4.189:38111)
0811 06:28:59.948489 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:28:59.948495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:28:59.967872 (+ 19377us) server_negotiation.cc:407] Connection header received
0811 06:28:59.968805 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:28:59.968811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:28:59.968894 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:28:59.968967 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:28:59.975534 (+ 6567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.976623 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.979767 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:28:59.980452 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:28:59.980545 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:28:59.993477 (+ 12932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:28:59.993483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:28:59.993491 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:28:59.993554 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:28:59.997549 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.998880 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.998888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.998894 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:28:59.999034 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:28:59.999895 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:28:59.999900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:28:59.999903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:00.000320 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:00.000586 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.001451 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.001724 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:29:00.003361 32485 log.cc:824] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:00.015319 32485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: No bootstrap required, opened a new log
I20260811 06:29:00.022209 32576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807d1b417c364257b720879ab6f8d1b6. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.022562 32576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 06:29:00.023029 32517 tablet.cc:2369] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:00.026070 32620 mvcc.cc:205] Tried to move back new op lower bound from 7317216214986973184 to 7317216214681268224. Current Snapshot: MvccSnapshot[applied={T|T < 7317216214986973184}]
I20260811 06:29:00.026911 32485 raft_consensus.cc:348] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:29:00.027433 32485 raft_consensus.cc:729] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a845f715b84e42c887d09e3df234ce12, State: Initialized, Role: FOLLOWER
I20260811 06:29:00.027856 32621 mvcc.cc:205] Tried to move back new op lower bound from 7317216214986973184 to 7317216214681268224. Current Snapshot: MvccSnapshot[applied={T|T < 7317216214986973184}]
I20260811 06:29:00.027989 32485 consensus_queue.cc:260] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } }
I20260811 06:29:00.053375 32592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.053755 32592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:00.059149 32608 catalog_manager.cc:1347] Generated new cluster ID: 073f1fb5098a445fb5a55238178adb60
I20260811 06:29:00.059331 32608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:00.060431 32576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.060706 32576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:00.062919 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.063494 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:00.064116 32485 sys_catalog.cc:564] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:00.064179 32458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "807d1b417c364257b720879ab6f8d1b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a845f715b84e42c887d09e3df234ce12"
I20260811 06:29:00.064651 32458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:00.068246 32458 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807d1b417c364257b720879ab6f8d1b6 in term 1.
I20260811 06:29:00.069290 32457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [term 1 FOLLOWER]: Refusing update from remote peer 807d1b417c364257b720879ab6f8d1b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:00.070502 32598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:00.089249 32608 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:00.091351 32608 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:00.121647 32608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6: Generated new TSK 0
I20260811 06:29:00.122771 32608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:29:00.132143 32626 sys_catalog.cc:455] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807d1b417c364257b720879ab6f8d1b6. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.132480 32626 sys_catalog.cc:458] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:00.132941 32637 sys_catalog.cc:455] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.133345 32637 sys_catalog.cc:458] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:00.134567 32598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "807d1b417c364257b720879ab6f8d1b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807d1b417c364257b720879ab6f8d1b6" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 34127 } } peers { permanent_uuid: "a845f715b84e42c887d09e3df234ce12" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 38111 } } peers { permanent_uuid: "0bc8e44f5baa47d3b04d867f6be2ed0e" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 46095 } } }
I20260811 06:29:00.134825 32598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807d1b417c364257b720879ab6f8d1b6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:00.143242 32644 catalog_manager.cc:1259] Loaded cluster ID: 073f1fb5098a445fb5a55238178adb60
I20260811 06:29:00.145008 32644 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: loading cluster ID for follower catalog manager: success
I20260811 06:29:00.173570 32644 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: acquiring CA information for follower catalog manager: success
I20260811 06:29:00.184685 32644 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a845f715b84e42c887d09e3df234ce12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:29:00.315260 32572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.300969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.7.4.190:34127)
0811 06:29:00.301178 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.301184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.312358 (+ 11174us) server_negotiation.cc:407] Connection header received
0811 06:29:00.313680 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.313687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.313780 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.313853 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.314979 (+ 1126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34934: BlockingRecv error: recv got EOF from 127.0.0.1:34934 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 06:29:00.318698 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.308149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.7.4.188:46095)
0811 06:29:00.308339 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.308348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.316601 (+ 8253us) server_negotiation.cc:407] Connection header received
0811 06:29:00.317312 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.317317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.317398 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.317467 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.318451 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39832: BlockingRecv error: recv got EOF from 127.0.0.1:39832 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 06:29:00.321853 32491 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.312879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.7.4.189:38111)
0811 06:29:00.313039 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.313043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.319918 (+ 6875us) server_negotiation.cc:407] Connection header received
0811 06:29:00.320571 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.320578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.320667 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.320731 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.321644 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43240: BlockingRecv error: recv got EOF from 127.0.0.1:43240 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:29:00.385138 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.365687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.7.4.189:38111)
0811 06:29:00.365904 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.365912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.376764 (+ 10852us) server_negotiation.cc:407] Connection header received
0811 06:29:00.377322 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.377331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.377610 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.377757 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.378530 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.379794 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.382098 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.382727 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.382839 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.383653 (+ 814us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.384334 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.384450 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.384714 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.384803 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 06:29:00.434021 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.418217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.7.4.188:46095)
0811 06:29:00.418422 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.418433 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.431914 (+ 13481us) server_negotiation.cc:407] Connection header received
0811 06:29:00.432262 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.432269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.432402 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.432501 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.433722 (+ 1221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39846: BlockingRecv error: recv got EOF from 127.0.0.1:39846 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:29:00.487354 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.468874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.7.4.189:38111)
0811 06:29:00.469086 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.469092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.479729 (+ 10637us) server_negotiation.cc:407] Connection header received
0811 06:29:00.480371 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.480378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.480501 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.480589 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.481434 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.482589 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.484723 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.485507 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.485610 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.486373 (+ 763us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.486564 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.486642 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.486912 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.487006 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 06:29:00.538120 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.524996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.7.4.188:46095)
0811 06:29:00.525235 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.525241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.535915 (+ 10674us) server_negotiation.cc:407] Connection header received
0811 06:29:00.536502 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.536509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.536615 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.536713 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.537830 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39862: BlockingRecv error: recv got EOF from 127.0.0.1:39862 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:29:00.592741 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.574702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.7.4.189:38111)
0811 06:29:00.574881 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.574887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.585596 (+ 10709us) server_negotiation.cc:407] Connection header received
0811 06:29:00.586089 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.586097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.586217 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.586288 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.587005 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.588271 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.590320 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.591064 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.591168 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.591841 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.591987 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.592072 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.592321 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.592420 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 06:29:00.633420 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.620476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.7.4.188:46095)
0811 06:29:00.620693 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.620701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.631499 (+ 10798us) server_negotiation.cc:407] Connection header received
0811 06:29:00.631955 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.631961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.632067 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.632139 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.633168 (+ 1029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39870: BlockingRecv error: recv got EOF from 127.0.0.1:39870 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:29:00.647758 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.641762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.7.4.189:38111)
0811 06:29:00.642005 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.642013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.642546 (+ 533us) server_negotiation.cc:407] Connection header received
0811 06:29:00.642757 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.642763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.642851 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.642959 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.643483 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.644246 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.645854 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.646431 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.646580 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.646969 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.647080 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.647186 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.647362 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.647443 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 06:29:00.656265 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.653743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39878 (local address 127.7.4.188:46095)
0811 06:29:00.653991 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.653997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.654684 (+ 687us) server_negotiation.cc:407] Connection header received
0811 06:29:00.654768 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.654776 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.654875 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.654959 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.656064 (+ 1105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39878: BlockingRecv error: recv got EOF from 127.0.0.1:39878 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 06:29:00.673483 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.667707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.7.4.189:38111)
0811 06:29:00.667927 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.667934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.668188 (+ 254us) server_negotiation.cc:407] Connection header received
0811 06:29:00.668350 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.668362 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.668449 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.668582 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.668983 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.670162 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.671456 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.672197 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.672336 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.672669 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.672831 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.672933 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.673061 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.673146 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 06:29:00.681702 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.679530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.7.4.188:46095)
0811 06:29:00.679766 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.679772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.680288 (+ 516us) server_negotiation.cc:407] Connection header received
0811 06:29:00.680473 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.680480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.680556 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.680661 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.681467 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39880: BlockingRecv error: recv got EOF from 127.0.0.1:39880 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:29:00.706768 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.701030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.7.4.189:38111)
0811 06:29:00.701223 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.701230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.701634 (+ 404us) server_negotiation.cc:407] Connection header received
0811 06:29:00.701788 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.701794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.701886 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.702003 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.702514 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.703688 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.704906 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.705586 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.705719 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.706068 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.706194 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.706281 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.706417 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.706494 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 06:29:00.714926 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.713157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.7.4.188:46095)
0811 06:29:00.713328 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.713334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.713687 (+ 353us) server_negotiation.cc:407] Connection header received
0811 06:29:00.713793 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.713798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.713893 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.713996 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.714715 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39892: BlockingRecv error: recv got EOF from 127.0.0.1:39892 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:29:00.754873 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.750162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.7.4.189:38111)
0811 06:29:00.750334 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.750339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.750763 (+ 424us) server_negotiation.cc:407] Connection header received
0811 06:29:00.750939 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.750948 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.751042 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.751134 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.751561 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.752239 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.753464 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.753929 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.754019 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.754334 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.754425 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.754486 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.754610 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.754669 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:29:00.762641 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.760654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.7.4.188:46095)
0811 06:29:00.760884 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.760892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.761220 (+ 328us) server_negotiation.cc:407] Connection header received
0811 06:29:00.761450 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.761454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.761523 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.761653 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.762400 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39896: BlockingRecv error: recv got EOF from 127.0.0.1:39896 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:29:00.834849 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.830029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.7.4.189:38111)
0811 06:29:00.830212 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.830217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.830584 (+ 367us) server_negotiation.cc:407] Connection header received
0811 06:29:00.830750 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.830754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.830824 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.830919 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.831338 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.832102 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.833447 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.833854 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.833941 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.834286 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.834384 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.834446 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.834596 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.834656 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:29:00.842367 32571 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:00.840686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.7.4.188:46095)
0811 06:29:00.840847 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.840851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.841237 (+ 386us) server_negotiation.cc:407] Connection header received
0811 06:29:00.841383 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.841387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.841460 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.841573 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:00.842207 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39912: BlockingRecv error: recv got EOF from 127.0.0.1:39912 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:29:00.892277 32590 catalog_manager.cc:1259] Loaded cluster ID: 073f1fb5098a445fb5a55238178adb60
I20260811 06:29:00.892377 32590 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: loading cluster ID for follower catalog manager: success
I20260811 06:29:00.894737 32590 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: acquiring CA information for follower catalog manager: success
I20260811 06:29:00.896618 32590 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bc8e44f5baa47d3b04d867f6be2ed0e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:29:00.979912 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.974160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.7.4.189:38111)
0811 06:29:00.974426 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.974435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.974764 (+ 329us) server_negotiation.cc:407] Connection header received
0811 06:29:00.975022 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.975026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.975100 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.975230 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.975742 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.976512 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.978043 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.978495 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.978664 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.979088 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.979208 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.979348 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.979513 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.979609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:29:00.991183 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.985719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.7.4.188:46095)
0811 06:29:00.985954 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.985959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.986141 (+ 182us) server_negotiation.cc:407] Connection header received
0811 06:29:00.986297 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.986300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.986456 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.986552 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:00.986941 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.987707 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.989130 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:00.989528 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:00.989622 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:00.990073 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:00.990579 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:00.990690 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:00.990791 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 06:29:00.990854 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 06:29:01.005281 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.997623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.7.4.188:46095)
0811 06:29:00.997836 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.997843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.998942 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 06:29:00.999251 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.999260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.999383 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.999477 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:01.000032 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.001096 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.003102 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.003730 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.003829 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:01.004496 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:01.004650 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:01.004715 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:01.004927 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:29:01.005001 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:29:01.006264 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.997442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.7.4.190:34127)
0811 06:29:00.998245 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.998254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:00.999236 (+ 982us) server_negotiation.cc:407] Connection header received
0811 06:29:00.999388 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:00.999395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:00.999621 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:00.999821 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:01.000502 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.001525 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.003631 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.004135 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.004273 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:01.004750 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:01.005179 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:01.005304 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:01.005830 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 06:29:01.005925 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":529,"threads_started":1}
I20260811 06:29:01.006544 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:00.998808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.7.4.189:38111)
0811 06:29:00.999007 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:00.999013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:01.000251 (+ 1238us) server_negotiation.cc:407] Connection header received
0811 06:29:01.000445 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:01.000450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:01.000528 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:01.000630 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:01.001144 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.002097 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.003869 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:01.004502 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:01.004636 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:01.005822 (+ 1186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:01.005938 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:01.006025 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:01.006204 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 06:29:01.006286 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:29:01.010701 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32330
I20260811 06:29:01.012027 32399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:01.121110 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 32330
W20260811 06:29:01.146329 32432 connection.cc:535] server connection from 127.0.0.1:43214 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:29:01.146934 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32411
I20260811 06:29:01.148069 32480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:01.254678 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 32411
I20260811 06:29:01.278568 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 32495
I20260811 06:29:01.279752 32562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:01.387989 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 32495
2026-08-11T06:29:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4656 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:01 dist-test-slave-07c2 krb5kdc[312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:01 dist-test-slave-07c2 krb5kdc[312](info): set up 2 sockets
Aug 11 06:29:01 dist-test-slave-07c2 krb5kdc[312](info): commencing operation
krb5kdc: starting...
W20260811 06:29:02.610625 7186 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/test-user.keytab.
Aug 11 06:29:02 dist-test-slave-07c2 krb5kdc[312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.4.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
Entry for principal kudu/127.7.4.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab.
I20260811 06:29:02.924170 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46391
--webserver_interface=127.7.4.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:03.262567 325 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:29:03.262867 325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:03.262933 325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:03.263011 325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:03.269979 325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:03.270056 325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:03.270112 325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:03.270153 325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:03.270207 325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:03.279330 325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.190.keytab
--principal=kudu/127.7.4.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.190:46391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.4.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:03.280431 325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:03.281648 325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:03.292933 334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:03.293066 331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:03.292992 332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:03.293814 325 server_base.cc:1034] running on GCE node
I20260811 06:29:03.294524 325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:03.295810 325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:03.297056 325 hybrid_clock.cc:648] HybridClock initialized: now 1786429743296969 us; error 32 us; skew 500 ppm
I20260811 06:29:03.297379 325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:03 dist-test-slave-07c2 krb5kdc[312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:03.300519 325 init.cc:377] Logged in from keytab as kudu/127.7.4.190@KRBTEST.COM (short username kudu)
I20260811 06:29:03.301843 325 webserver.cc:460] Webserver started at http://127.7.4.190:44205/ using document root <none> and password file <none>
I20260811 06:29:03.302500 325 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:03.302618 325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:03.302964 325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:03.305809 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data/instance:
uuid: "f7e2bdf9e910445a96a80c16da6cf021"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "2d8c65277e2911acd26ae3628d4ab63b"
server_key_iv: "a3b007371fcac131b481cfb96a2bb196"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.306548 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance:
uuid: "f7e2bdf9e910445a96a80c16da6cf021"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "2d8c65277e2911acd26ae3628d4ab63b"
server_key_iv: "a3b007371fcac131b481cfb96a2bb196"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.311549 325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:29:03.314800 341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:03.316041 325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:29:03.316231 325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal
uuid: "f7e2bdf9e910445a96a80c16da6cf021"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "2d8c65277e2911acd26ae3628d4ab63b"
server_key_iv: "a3b007371fcac131b481cfb96a2bb196"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.316453 325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:03.345506 325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:03.349329 325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:03.349597 325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:03.368091 325 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.190:46391
I20260811 06:29:03.368120 395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.190:46391 every 8 connection(s)
I20260811 06:29:03.369408 325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:03.372891 396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:03.377971 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 325
I20260811 06:29:03.378181 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-0/wal/instance
I20260811 06:29:03.378738 7186 external_mini_cluster.cc:1428] Setting key 07a64f0d54033b86f840c948a7609c11
I20260811 06:29:03.379920 396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:03 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:29:03.402484 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.383829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.7.4.190:46391)
0811 06:29:03.385470 (+ 1641us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:03.385478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:03.385532 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:29:03.387024 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:03.387029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:03.387237 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:03.387491 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:03.388776 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.389937 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.391312 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.391842 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.392045 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.395650 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:03.395657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:03.395667 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:03.395694 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:03.398330 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.399007 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.399017 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.399020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.399164 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.399607 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.399614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.399617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.400049 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:03.400261 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:03.400780 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 06:29:03.401064 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":1425,"thread_start_us":1214,"threads_started":1}
I20260811 06:29:03.402522 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.383816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46391 (local address 127.0.0.1:50576)
0811 06:29:03.384681 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.384725 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:03.386675 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:03.387496 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:03.387502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:03.388227 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:03.388552 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.388569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.390090 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.390095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.391131 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.391153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.392073 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.392080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.392301 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.393019 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:03.393042 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:03.395428 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:03.398467 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.398480 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.398490 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.398877 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.399273 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.399279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.399283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.399500 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.400196 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:03.400204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:03.400388 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 06:29:03.400615 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 06:29:03.401191 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":156,"thread_start_us":406,"threads_started":1}
I20260811 06:29:03.405413 396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:29:03.406447 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.406053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34338)
0811 06:29:03.406242 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.406337 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 06:29:03.406716 343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
I20260811 06:29:03.408360 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.407877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34354)
0811 06:29:03.408081 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.408189 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 06:29:03.408610 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:33421: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.4.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
Entry for principal kudu/127.7.4.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab.
I20260811 06:29:03.454409 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:33421
--webserver_interface=127.7.4.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:29:03.458043 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } attempt: 1
I20260811 06:29:03.459233 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.458797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34356)
0811 06:29:03.458994 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.459098 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:29:03.460397 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.459931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34364)
0811 06:29:03.460106 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.460216 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:29:03.460597 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:33421: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
I20260811 06:29:03.528887 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } attempt: 2
I20260811 06:29:03.530069 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.529576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34368)
0811 06:29:03.529839 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.529932 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:29:03.531242 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.530808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34370)
0811 06:29:03.531021 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.531094 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:29:03.531502 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:33421: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
I20260811 06:29:03.622781 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } attempt: 3
I20260811 06:29:03.624053 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.623574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34382)
0811 06:29:03.623783 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.623903 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 06:29:03.625087 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.624691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34386)
0811 06:29:03.624871 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.624951 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:29:03.625288 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:33421: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
I20260811 06:29:03.768517 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } attempt: 4
I20260811 06:29:03.769760 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.769218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34388)
0811 06:29:03.769541 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.769629 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 06:29:03.770750 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.770393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34402)
0811 06:29:03.770541 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.770627 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:29:03.770938 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.189:33421: Network error: Client connection negotiation failed: client connection to 127.7.4.189:33421: connect: Connection refused (error 111)
W20260811 06:29:03.797127 404 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:29:03.797488 404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:03.797632 404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:03.797770 404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:03.804934 404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:03.805054 404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:03.805167 404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:03.805256 404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:03.805346 404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:03.815223 404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.189.keytab
--principal=kudu/127.7.4.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.189:33421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.4.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:03.816571 404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:03.817910 404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:03.829496 410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:03.829499 411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:03.829440 413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:03.832741 404 server_base.cc:1034] running on GCE node
I20260811 06:29:03.833458 404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:03.834748 404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:03.836023 404 hybrid_clock.cc:648] HybridClock initialized: now 1786429743835963 us; error 45 us; skew 500 ppm
I20260811 06:29:03.836356 404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:03 dist-test-slave-07c2 krb5kdc[312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:03.839612 404 init.cc:377] Logged in from keytab as kudu/127.7.4.189@KRBTEST.COM (short username kudu)
I20260811 06:29:03.841161 404 webserver.cc:460] Webserver started at http://127.7.4.189:40535/ using document root <none> and password file <none>
I20260811 06:29:03.841815 404 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:03.841903 404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:03.842171 404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:03.845082 404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data/instance:
uuid: "b1ee6389264e4867bdd91e2a48ad60fd"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "ab7e3954503b1d54f125f4dc57179a72"
server_key_iv: "e418de4f2d4825b56be52d5c16b6187b"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.845849 404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal/instance:
uuid: "b1ee6389264e4867bdd91e2a48ad60fd"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "ab7e3954503b1d54f125f4dc57179a72"
server_key_iv: "e418de4f2d4825b56be52d5c16b6187b"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.850809 404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:03.854118 420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:03.855511 404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:03.855664 404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal
uuid: "b1ee6389264e4867bdd91e2a48ad60fd"
format_stamp: "Formatted at 2026-08-11 06:29:03 on dist-test-slave-07c2"
server_key: "ab7e3954503b1d54f125f4dc57179a72"
server_key_iv: "e418de4f2d4825b56be52d5c16b6187b"
server_key_version: "encryptionkey@0"
I20260811 06:29:03.855861 404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:03.871995 404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:03.876304 404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:03.876531 404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:03.895601 404 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.189:33421
I20260811 06:29:03.895633 484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.189:33421 every 8 connection(s)
I20260811 06:29:03.896910 404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/data/info.pb
I20260811 06:29:03.900465 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:03.907076 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 404
I20260811 06:29:03.907279 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-1/wal/instance
I20260811 06:29:03.907856 7186 external_mini_cluster.cc:1428] Setting key 8154137e7a11377edb0fdef67d3db058
I20260811 06:29:03.907044 485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:03 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:29:03.929106 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.910181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.7.4.190:46391)
0811 06:29:03.910713 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:03.910722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:03.911690 (+ 968us) server_negotiation.cc:407] Connection header received
0811 06:29:03.913564 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:03.913572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:03.913698 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:03.913835 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:03.915277 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.916407 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.917882 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.918577 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.918733 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.922095 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:03.922104 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:03.922111 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:03.922172 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:03.925784 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.926698 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.926702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.926704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.926820 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.927342 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.927351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.927353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.927608 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:03.927764 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:03.928683 (+ 919us) server_negotiation.cc:299] Negotiation successful
0811 06:29:03.928844 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":250,"threads_started":1}
I20260811 06:29:03.930622 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.910706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46391 (local address 127.0.0.1:50586)
0811 06:29:03.911516 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.911569 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:03.913299 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:03.914027 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:03.914034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:03.914540 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:03.915067 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.915084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.916567 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.916570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.917724 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.917736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.918783 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.918787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.918973 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.919714 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:03.919735 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:03.921878 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:03.925928 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.925939 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.925955 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.926579 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.926929 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.926936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.926943 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.927188 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.927765 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:03.927783 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:03.928226 (+ 443us) client_negotiation.cc:769] Sending connection context
0811 06:29:03.928604 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 06:29:03.929143 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":356,"threads_started":1}
I20260811 06:29:03.932423 485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:03 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.4.188@KRBTEST.COM; defaulting to no policy
I20260811 06:29:03.948503 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.933088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34406)
0811 06:29:03.933396 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.933416 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:03.933517 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:03.934466 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:03.934473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:03.934980 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:03.935188 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.935202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.936604 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.936612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.938169 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:03.938187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.939046 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.939055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:03.939272 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.939890 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:03.939917 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:03.942460 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:03.945925 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.945930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.945934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.946315 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.946682 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:03.946688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:03.946695 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:03.946830 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:03.947511 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:03.947520 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:03.947661 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 06:29:03.947809 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 06:29:03.948091 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 06:29:03.948625 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.933327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.7.4.189:33421)
0811 06:29:03.934066 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:03.934072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:03.934097 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:29:03.934139 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:03.934145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:03.934297 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:03.934408 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:03.935400 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.936426 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.938315 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:03.938863 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:03.938957 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:03.942800 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:03.942812 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:03.942825 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:03.942855 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:03.945774 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.946443 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.946457 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.946459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.946568 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:03.946930 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:03.946937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:03.946941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:03.947371 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:03.947591 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:03.947955 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:29:03.948209 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":597,"thread_start_us":449,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.4.188@KRBTEST.COM" created.
I20260811 06:29:03.951248 485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:29:03.952587 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.952013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:33982)
0811 06:29:03.952237 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.952396 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:29:03.952934 424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:03.954571 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:03.954142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:33996)
0811 06:29:03.954332 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:03.954431 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:29:03.954798 485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
Entry for principal kudu/127.7.4.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab.
I20260811 06:29:03.990298 7186 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5M28Km/build/asan/bin/kudu
/tmp/dist-test-task5M28Km/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:34515
--webserver_interface=127.7.4.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.4.148:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:29:04.004106 485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 1
I20260811 06:29:04.005278 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.004835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34000)
0811 06:29:04.005031 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.005119 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:29:04.006266 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.005920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34008)
0811 06:29:04.006060 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.006160 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:29:04.006464 485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.070183 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } attempt: 5
I20260811 06:29:04.074673 485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 2
I20260811 06:29:04.075603 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.075198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34012)
0811 06:29:04.075361 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.075457 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:29:04.076771 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.076362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34016)
0811 06:29:04.076527 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.076614 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:29:04.077033 485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:29:04.084321 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.071096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.7.4.189:33421)
0811 06:29:04.071355 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.071362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.071385 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:29:04.071478 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.071483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.071555 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.071668 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.072463 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.073170 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.074356 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.074957 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.075085 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.079115 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.079120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.079123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.079154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.081758 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.082528 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.082532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.082534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.082652 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.083049 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.083053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.083054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.083385 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.083498 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.083968 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.084122 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:29:04.084652 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.070966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34418)
0811 06:29:04.071177 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.071202 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.071360 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.071697 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.071702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.072115 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.072303 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.072312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.073321 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.073329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.074202 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.074212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.075149 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.075158 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.075485 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.076218 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.076246 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.078934 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.081898 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.081903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.081907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.082418 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.082762 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.082767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.082770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.082938 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.083505 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.083510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.083673 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.083870 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.084196 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 06:29:04.085655 396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:29:04.086766 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.086226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34032)
0811 06:29:04.086440 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.086549 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 06:29:04.087801 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.087434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34038)
0811 06:29:04.087589 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.087668 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:29:04.088019 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.139309 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 1
I20260811 06:29:04.140533 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.139947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34054)
0811 06:29:04.140160 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.140291 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:29:04.141637 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.141203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34062)
0811 06:29:04.141388 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.141487 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:29:04.141863 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.168265 485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 3
I20260811 06:29:04.169405 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.168961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34074)
0811 06:29:04.169167 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.169257 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:29:04.170548 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.170163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34078)
0811 06:29:04.170371 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.170441 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260811 06:29:04.170964 485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.210108 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 2
I20260811 06:29:04.211570 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.210915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34092)
0811 06:29:04.211187 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.211343 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:29:04.212644 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.212254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34094)
0811 06:29:04.212422 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.212512 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 06:29:04.212877 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.314208 485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 4
I20260811 06:29:04.315495 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.314967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34108)
0811 06:29:04.315196 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.315327 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 06:29:04.316427 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.316150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34114)
0811 06:29:04.316271 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.316332 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 06:29:04.316682 485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
I20260811 06:29:04.319116 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 3
I20260811 06:29:04.320258 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.319737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34120)
0811 06:29:04.319969 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.320069 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 06:29:04.321358 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.320861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34132)
0811 06:29:04.321130 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.321222 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260811 06:29:04.321570 396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.4.188:34515: Network error: Client connection negotiation failed: client connection to 127.7.4.188:34515: connect: Connection refused (error 111)
W20260811 06:29:04.334857 494 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:29:04.335199 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:04.335333 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:04.335471 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:04.342610 494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:04.342687 494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:04.342795 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:04.342880 494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:04.342969 494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:04.352661 494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.4.148:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.4.190:46391,127.7.4.189:33421,127.7.4.188:34515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/krb5kdc/kudu_127.7.4.188.keytab
--principal=kudu/127.7.4.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.4.188:34515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.4.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:04.353906 494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:04.355255 494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:04.367429 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:04.367404 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:04.367662 503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:04.368546 494 server_base.cc:1034] running on GCE node
I20260811 06:29:04.369477 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:04.371038 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:04.372311 494 hybrid_clock.cc:648] HybridClock initialized: now 1786429744372230 us; error 45 us; skew 500 ppm
I20260811 06:29:04.372684 494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:04.376609 494 init.cc:377] Logged in from keytab as kudu/127.7.4.188@KRBTEST.COM (short username kudu)
I20260811 06:29:04.378217 494 webserver.cc:460] Webserver started at http://127.7.4.188:42841/ using document root <none> and password file <none>
I20260811 06:29:04.378903 494 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:04.379038 494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:04.379446 494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:04.382467 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data/instance:
uuid: "1029823f8bba421eb65fc07fcc032f2f"
format_stamp: "Formatted at 2026-08-11 06:29:04 on dist-test-slave-07c2"
server_key: "56873e0d4314edc013c483cf2b82c1c3"
server_key_iv: "3a87bf8a5446cfa6bb3725e38b202b64"
server_key_version: "encryptionkey@0"
I20260811 06:29:04.383252 494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal/instance:
uuid: "1029823f8bba421eb65fc07fcc032f2f"
format_stamp: "Formatted at 2026-08-11 06:29:04 on dist-test-slave-07c2"
server_key: "56873e0d4314edc013c483cf2b82c1c3"
server_key_iv: "3a87bf8a5446cfa6bb3725e38b202b64"
server_key_version: "encryptionkey@0"
I20260811 06:29:04.388425 494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:29:04.391757 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:04.392946 494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:29:04.393132 494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data,/tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal
uuid: "1029823f8bba421eb65fc07fcc032f2f"
format_stamp: "Formatted at 2026-08-11 06:29:04 on dist-test-slave-07c2"
server_key: "56873e0d4314edc013c483cf2b82c1c3"
server_key_iv: "3a87bf8a5446cfa6bb3725e38b202b64"
server_key_version: "encryptionkey@0"
I20260811 06:29:04.393349 494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:04.406435 494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:04.410745 494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:04.411029 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:04.430246 494 rpc_server.cc:307] RPC server started. Bound to: 127.7.4.188:34515
I20260811 06:29:04.430289 577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.4.188:34515 every 8 connection(s)
I20260811 06:29:04.431687 494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/data/info.pb
I20260811 06:29:04.432303 7186 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5M28Km/build/asan/bin/kudu as pid 494
I20260811 06:29:04.432540 7186 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5M28Km/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429373903770-7186-0/minicluster-data/master-2/wal/instance
I20260811 06:29:04.433138 7186 external_mini_cluster.cc:1428] Setting key 7cad1427693ec7ea39eea9e501a8ebe9
I20260811 06:29:04.436363 579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:04.445156 579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
I20260811 06:29:04.458195 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.434977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.7.4.188:34515)
0811 06:29:04.435840 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.435855 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.435907 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:29:04.438185 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.438213 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.438720 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.439128 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.440483 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.442548 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.443855 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.444910 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.445094 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.449606 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.449623 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.449636 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.449691 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.452899 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.453839 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.453850 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.453860 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.454139 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.454647 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.454652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.454655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.455443 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.455806 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.456166 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.456674 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":367,"threads_started":1}
I20260811 06:29:04.464511 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.447600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.190:46391 (local address 127.0.0.1:50596)
0811 06:29:04.448485 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.448524 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.448714 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.449109 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.449117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.449999 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.450394 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.450405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.451489 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.451494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.452597 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.452609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.453531 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.453536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.453972 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.454751 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.454784 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.457936 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.461530 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.461544 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.461555 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.462168 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.462709 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.462713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.462715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.462836 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.463518 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.463523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.463683 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.463830 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.464034 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"spinlock_wait_cycles":17152,"thread_start_us":542,"threads_started":1}
I20260811 06:29:04.464670 584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.447588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.7.4.190:46391)
0811 06:29:04.448166 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.448174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.448726 (+ 552us) server_negotiation.cc:407] Connection header received
0811 06:29:04.448847 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.448852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.448926 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.449039 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.450538 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.451286 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.452780 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.453350 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.453500 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.458159 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.458164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.458166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.458207 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.461367 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.462310 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.462315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.462318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.462529 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.462964 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.462969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.462973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.463389 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.463609 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.463986 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.464266 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":288,"threads_started":1}
I20260811 06:29:04.466058 579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.190@KRBTEST.COM
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
I20260811 06:29:04.477830 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.461489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.7.4.190:46391)
0811 06:29:04.462182 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.462188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.462212 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:29:04.462263 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.462268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.462339 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.462441 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.464181 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.465336 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.466811 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.467393 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.467534 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.471892 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.471897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.471900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.471929 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.475063 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.475967 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.475975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.475980 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.476114 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.476504 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.476511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.476514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.476862 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.477138 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.477348 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.477515 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":278,"threads_started":1}
I20260811 06:29:04.480310 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.466671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.189:33421 (local address 127.0.0.1:34428)
0811 06:29:04.466845 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.466863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.466947 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.467350 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.467357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.467782 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.467915 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.467934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.468894 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.468900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.469786 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.469796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.470575 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.470580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.470709 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.471345 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.471375 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.473602 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.477338 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.477344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.477348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.477909 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.478454 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.478461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.478464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.478671 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.479256 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.479261 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.479460 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.479619 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.479910 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:29:04.480868 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.466856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.7.4.189:33421)
0811 06:29:04.467029 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.467034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.467057 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:29:04.467110 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.467114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.467172 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.467252 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.468078 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.468742 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.469925 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.470385 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.470525 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.473796 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.473802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.473807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.473849 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.477199 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.478047 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.478053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.478058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.478284 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.478798 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.478801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.478803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.479127 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.479238 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.480231 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.480474 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:29:04.481748 579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.4.189@KRBTEST.COM
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429744, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.188@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:29:04.495975 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.479706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.7.4.189:33421)
0811 06:29:04.480272 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.480276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.480295 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:29:04.480331 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.480334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.480413 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.480521 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.482281 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.483226 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.484446 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.484879 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.485017 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.489403 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.489412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.489417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.489458 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.492933 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.493721 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.493728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.493734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.493893 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.494306 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.494313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.494319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.494656 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.494951 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.495215 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.495488 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":277,"threads_started":1}
I20260811 06:29:04.496533 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.482654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.7.4.188:34515)
0811 06:29:04.482911 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.482917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.482934 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:29:04.483018 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.483023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.483100 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.483204 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.483985 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.484735 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.486215 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.486682 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.486834 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.490627 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.490631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.490634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.490673 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.493481 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.494368 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.494372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.494374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.494495 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.494921 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.494925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.494928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.495251 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.495407 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.495859 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.496269 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":160}
I20260811 06:29:04.497004 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.482624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34152)
0811 06:29:04.482790 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.482811 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.482913 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.483225 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.483230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.483706 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.483838 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.483846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.484869 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.484877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.485986 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.485998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.486851 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.486858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.487058 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.487709 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.487737 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.490460 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.493636 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.493639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.493642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.494219 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.494615 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.494621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.494625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.494800 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.495398 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.495404 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.495561 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.495741 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.496422 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:29:04.498780 396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 4
I20260811 06:29:04.501799 7186 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:29:04.504431 579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: Bootstrap starting.
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.190@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:29:04.504114 371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f7e2bdf9e910445a96a80c16da6cf021"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50612
I20260811 06:29:04.508147 579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:04.509511 579 log.cc:824] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:04.513329 579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: No bootstrap required, opened a new log
I20260811 06:29:04.515508 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.499444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.7.4.188:34515)
0811 06:29:04.499865 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.499892 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.499910 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:29:04.499956 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.499960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.500025 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.500126 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.501040 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.501949 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.503817 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.504527 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.504671 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.508903 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.508909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.508913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.508951 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.512310 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.513132 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.513137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.513140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.513295 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.513844 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.513852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.513856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.514249 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.514421 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.514870 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.515100 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260811 06:29:04.515566 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.499379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34160)
0811 06:29:04.499598 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.499612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.499697 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.500181 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.500188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.500683 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.500889 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.500901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.502106 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.502113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.503628 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.503647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.504713 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.504721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.504956 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.505600 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.505629 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.508706 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.512444 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.512451 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.512454 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.513001 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.513432 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.513437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.513441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.513664 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.514391 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.514397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.514586 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.514749 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.515089 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 06:29:04.520973 396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: Bootstrap starting.
I20260811 06:29:04.522913 579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.523370 579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1029823f8bba421eb65fc07fcc032f2f, State: Initialized, Role: FOLLOWER
I20260811 06:29:04.523927 396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:04.523917 579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.525233 396 log.cc:824] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:04.524950 591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.525403 591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.526155 579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:04.528173 396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: No bootstrap required, opened a new log
I20260811 06:29:04.534488 396 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.534952 396 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e2bdf9e910445a96a80c16da6cf021, State: Initialized, Role: FOLLOWER
I20260811 06:29:04.535531 396 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
W20260811 06:29:04.538182 606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:29:04.538444 606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:29:04.539898 608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.540376 608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.540977 396 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:04.541116 371 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:29:04.541359 371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:04.545182 371 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.548681 546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7e2bdf9e910445a96a80c16da6cf021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1029823f8bba421eb65fc07fcc032f2f"
I20260811 06:29:04.549177 546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:04.535507 494 master_runner.cc:186] Error getting master registration for 127.7.4.190:46391: OK, instance_id { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" instance_seqno: 1786429743362890 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:29:04.549981 494 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:29:04.552815 546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e2bdf9e910445a96a80c16da6cf021 in term 1.
I20260811 06:29:04.553407 371 leader_election.cc:288] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [CANDIDATE]: Term 1 election: Requested vote from peers b1ee6389264e4867bdd91e2a48ad60fd (127.7.4.189:33421), 1029823f8bba421eb65fc07fcc032f2f (127.7.4.188:34515)
I20260811 06:29:04.553884 345 leader_election.cc:302] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1029823f8bba421eb65fc07fcc032f2f, f7e2bdf9e910445a96a80c16da6cf021; no voters:
I20260811 06:29:04.554478 608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:04.557010 608 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [term 1 LEADER]: Becoming Leader. State: Replica: f7e2bdf9e910445a96a80c16da6cf021, State: Running, Role: LEADER
I20260811 06:29:04.557574 608 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.564785 625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e2bdf9e910445a96a80c16da6cf021. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.565392 625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: This master's current role is: LEADER
W20260811 06:29:04.565768 622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:29:04.565977 622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:29:04.566568 630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:04.568742 630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:04.561729 325 master_runner.cc:186] Error getting master registration for 127.7.4.189:33421: OK, instance_id { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" instance_seqno: 1786429743890177 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:29:04.588740 325 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:29:04.601424 546 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [term 1 FOLLOWER]: Refusing update from remote peer f7e2bdf9e910445a96a80c16da6cf021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 06:29:04.602129 347 tablet.cc:2369] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:04.602613 608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:04.613612 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.589435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.7.4.190:46391)
0811 06:29:04.589680 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.589688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.589725 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:29:04.590012 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.590017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.590110 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.590196 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.592811 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.594212 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.596575 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.597456 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.597580 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.605575 (+ 7995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.605582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.605586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.605624 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.609761 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.610765 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.610772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.610778 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.610933 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.611470 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.611477 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.611483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.611942 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.612229 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.612966 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.613231 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:29:04.615839 485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } attempt: 5
I20260811 06:29:04.632583 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.591408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.7.4.189:33421)
0811 06:29:04.591637 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.591644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.614739 (+ 23095us) server_negotiation.cc:407] Connection header received
0811 06:29:04.616448 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.616455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.616556 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.616640 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.618279 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.619373 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.621320 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.621985 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.622083 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.625407 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.625414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.625419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.625459 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.629226 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.630163 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.630168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.630172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.630324 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.630814 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.630819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.630823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.631180 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.631429 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.631998 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.632220 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:29:04 dist-test-slave-07c2 krb5kdc[312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429743, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.4.189@KRBTEST.COM for kudu/127.7.4.188@KRBTEST.COM
I20260811 06:29:04.641880 591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e2bdf9e910445a96a80c16da6cf021. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.642238 591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.643357 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.590388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.7.4.188:34515)
0811 06:29:04.590563 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.590570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.613688 (+ 23118us) server_negotiation.cc:407] Connection header received
0811 06:29:04.614984 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.614991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.615090 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.615164 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.619383 (+ 4219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.620613 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.626406 (+ 5793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.627159 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.627279 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.635943 (+ 8664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.635949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.635956 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.635992 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.639820 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.640644 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.640649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.640652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.640830 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.641319 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.641324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.641327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.641689 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.641942 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.642690 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.642940 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:29:04.650940 608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.653324 608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:04.656157 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.616417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.4.188:34515 (local address 127.0.0.1:34188)
0811 06:29:04.616772 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:04.616793 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:04.616901 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:04.629559 (+ 12658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:04.629565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:04.630120 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:04.630376 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.630391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.632876 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.632883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.634851 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:04.634868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.636289 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.636299 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:04.636598 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.637121 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:04.637147 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:04.640593 (+ 3446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:04.652854 (+ 12261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.652863 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.652870 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.653514 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.654103 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:04.654108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:04.654112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:04.654295 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:04.655043 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:04.655049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:04.655210 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 06:29:04.655406 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:29:04.655710 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260811 06:29:04.657665 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.617122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.7.4.188:34515)
0811 06:29:04.629172 (+ 12050us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.629181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.629208 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:29:04.629258 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.629263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.629339 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.630544 (+ 1205us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.630931 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.632679 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.635084 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.636076 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.636972 (+ 896us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.640965 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:04.640971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:04.640975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:04.641005 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:04.652690 (+ 11685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.653720 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.653728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.653732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.653926 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:04.654447 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:04.654452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:04.654455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:04.654834 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:04.656748 (+ 1914us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.656876 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.657069 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11871,"thread_start_us":11725,"threads_started":1}
I20260811 06:29:04.659576 630 catalog_manager.cc:1347] Generated new cluster ID: 1312a6697f4940cfab357a7fa7ac9de5
I20260811 06:29:04.659673 630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:04.666508 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: Bootstrap starting.
I20260811 06:29:04.675683 485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:04.677116 591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.677393 591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.681457 485 log.cc:824] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:04.703531 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: No bootstrap required, opened a new log
I20260811 06:29:04.704105 630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:04.706045 630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:04.714756 485 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.715137 485 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ee6389264e4867bdd91e2a48ad60fd, State: Initialized, Role: FOLLOWER
I20260811 06:29:04.715667 485 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } }
I20260811 06:29:04.717144 645 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.718061 485 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:04.719372 454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7e2bdf9e910445a96a80c16da6cf021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b1ee6389264e4867bdd91e2a48ad60fd"
I20260811 06:29:04.720038 453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:04.729287 453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [term 1 FOLLOWER]: Refusing update from remote peer f7e2bdf9e910445a96a80c16da6cf021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:04.731942 630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021: Generated new TSK 0
I20260811 06:29:04.732939 630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:29:04.732980 454 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e2bdf9e910445a96a80c16da6cf021 in term 1.
I20260811 06:29:04.735430 608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:04.736392 645 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.808964 645 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e2bdf9e910445a96a80c16da6cf021. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.808936 655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:04.809360 645 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.809432 655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:29:04.842288 666 catalog_manager.cc:1259] Loaded cluster ID: 1312a6697f4940cfab357a7fa7ac9de5
I20260811 06:29:04.842396 666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: loading cluster ID for follower catalog manager: success
I20260811 06:29:04.846024 666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: acquiring CA information for follower catalog manager: success
I20260811 06:29:04.848420 666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b1ee6389264e4867bdd91e2a48ad60fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:29:04.872982 7186 mini_kdc.cc:348] Time spent kdestroy: real 0.125s user 0.000s sys 0.033s
W20260811 06:29:04.933671 587 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:04.915595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.7.4.190:46391)
0811 06:29:04.915790 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.915797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.929421 (+ 13624us) server_negotiation.cc:407] Connection header received
0811 06:29:04.931543 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.931549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.931666 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.931744 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.933380 (+ 1636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50626: BlockingRecv error: recv got EOF from 127.0.0.1:50626 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 06:29:04.935495 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:04.924236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.7.4.188:34515)
0811 06:29:04.924430 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.924440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.931078 (+ 6638us) server_negotiation.cc:407] Connection header received
0811 06:29:04.931955 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.931961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.932069 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.932143 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.935250 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34200: BlockingRecv error: recv got EOF from 127.0.0.1:34200 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 06:29:04.939504 588 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:04.930225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.7.4.189:33421)
0811 06:29:04.930404 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.930410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.937157 (+ 6747us) server_negotiation.cc:407] Connection header received
0811 06:29:04.938138 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.938146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.938247 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.938320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:04.939242 (+ 922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34458: BlockingRecv error: recv got EOF from 127.0.0.1:34458 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:29:04.991916 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:04.971564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.7.4.189:33421)
0811 06:29:04.971753 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:04.971759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:04.984280 (+ 12521us) server_negotiation.cc:407] Connection header received
0811 06:29:04.984795 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:04.984803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:04.985008 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:04.985160 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:04.985915 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.987106 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.989126 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:04.989791 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:04.989874 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:04.990541 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:04.991147 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:04.991253 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:04.991500 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 06:29:04.991585 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 06:29:05.034850 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.026039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.7.4.188:34515)
0811 06:29:05.026240 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.026246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.032652 (+ 6406us) server_negotiation.cc:407] Connection header received
0811 06:29:05.033206 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.033213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.033343 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.033430 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.034552 (+ 1122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34210: BlockingRecv error: recv got EOF from 127.0.0.1:34210 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:29:05.097967 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.079825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.7.4.189:33421)
0811 06:29:05.080015 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.080022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.090592 (+ 10570us) server_negotiation.cc:407] Connection header received
0811 06:29:05.091175 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.091183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.091287 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.091400 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.092139 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.093318 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.095456 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.096172 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.096302 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.097000 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.097170 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.097243 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.097510 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.097628 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 06:29:05.140861 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.131603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.7.4.188:34515)
0811 06:29:05.131841 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.131848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.138706 (+ 6858us) server_negotiation.cc:407] Connection header received
0811 06:29:05.139237 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.139244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.139403 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.139489 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.140609 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34226: BlockingRecv error: recv got EOF from 127.0.0.1:34226 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:29:05.201598 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.183617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.7.4.189:33421)
0811 06:29:05.183816 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.183823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.194326 (+ 10503us) server_negotiation.cc:407] Connection header received
0811 06:29:05.194829 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.194836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.194928 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.194994 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.195807 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.197007 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.199244 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.199875 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.199978 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.200637 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.200804 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.200874 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.201145 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.201235 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 06:29:05.224080 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.221589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.7.4.188:34515)
0811 06:29:05.221780 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.221789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.222350 (+ 561us) server_negotiation.cc:407] Connection header received
0811 06:29:05.222526 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.222534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.222673 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.222802 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.223801 (+ 999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34238: BlockingRecv error: recv got EOF from 127.0.0.1:34238 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:29:05.244295 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.237318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.7.4.189:33421)
0811 06:29:05.237548 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.237555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.238095 (+ 540us) server_negotiation.cc:407] Connection header received
0811 06:29:05.238289 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.238299 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.238409 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.238514 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.238970 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.240144 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.242076 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.242761 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.242885 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.243443 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.243596 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.243692 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.243859 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.243941 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260811 06:29:05.253890 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.251175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.7.4.188:34515)
0811 06:29:05.251534 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.251541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.252123 (+ 582us) server_negotiation.cc:407] Connection header received
0811 06:29:05.252407 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.252413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.252505 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.252583 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.253635 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34244: BlockingRecv error: recv got EOF from 127.0.0.1:34244 (error 108)
Metrics: {"server-negotiator.queue_time_us":214}
I20260811 06:29:05.272804 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.265716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.7.4.189:33421)
0811 06:29:05.265880 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.265885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.266426 (+ 541us) server_negotiation.cc:407] Connection header received
0811 06:29:05.266617 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.266622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.266695 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.266784 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.267227 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.268238 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.270643 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.271341 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.271460 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.271926 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.272056 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.272145 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.272320 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.272404 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 06:29:05.281481 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.279426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.7.4.188:34515)
0811 06:29:05.279657 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.279664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.280050 (+ 386us) server_negotiation.cc:407] Connection header received
0811 06:29:05.280232 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.280235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.280303 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.280394 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.281256 (+ 862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34254: BlockingRecv error: recv got EOF from 127.0.0.1:34254 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:29:05.306603 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.300825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.7.4.189:33421)
0811 06:29:05.301022 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.301027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.301319 (+ 292us) server_negotiation.cc:407] Connection header received
0811 06:29:05.301491 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.301494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.301556 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.301630 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.302188 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.302980 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.304941 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.305333 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.305421 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.305917 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.306011 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.306078 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.306258 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.306319 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 06:29:05.319124 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.317055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.7.4.188:34515)
0811 06:29:05.317257 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.317262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.317573 (+ 311us) server_negotiation.cc:407] Connection header received
0811 06:29:05.317751 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.317755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.317830 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.317910 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.318872 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34268: BlockingRecv error: recv got EOF from 127.0.0.1:34268 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:29:05.354604 608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7e2bdf9e910445a96a80c16da6cf021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7e2bdf9e910445a96a80c16da6cf021" member_type: VOTER last_known_addr { host: "127.7.4.190" port: 46391 } } peers { permanent_uuid: "b1ee6389264e4867bdd91e2a48ad60fd" member_type: VOTER last_known_addr { host: "127.7.4.189" port: 33421 } } peers { permanent_uuid: "1029823f8bba421eb65fc07fcc032f2f" member_type: VOTER last_known_addr { host: "127.7.4.188" port: 34515 } } }
I20260811 06:29:05.354941 608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e2bdf9e910445a96a80c16da6cf021 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:05.362792 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.356224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.7.4.189:33421)
0811 06:29:05.356413 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.356419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.357497 (+ 1078us) server_negotiation.cc:407] Connection header received
0811 06:29:05.357566 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.357570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.357642 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.357727 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.358199 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.359063 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.361076 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.361491 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.361582 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.362028 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.362161 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.362226 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.362403 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.362462 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 06:29:05.373370 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.369216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.7.4.188:34515)
0811 06:29:05.369388 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.369394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.371853 (+ 2459us) server_negotiation.cc:407] Connection header received
0811 06:29:05.372007 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.372012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.372101 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.372200 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.373124 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34270: BlockingRecv error: recv got EOF from 127.0.0.1:34270 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:29:05.446590 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.440696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.7.4.189:33421)
0811 06:29:05.440953 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.440959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.441310 (+ 351us) server_negotiation.cc:407] Connection header received
0811 06:29:05.441497 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.441501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.441570 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.441650 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.442160 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.442985 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.444856 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.445236 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.445327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.445799 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.445925 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.445993 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.446153 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.446215 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 06:29:05.454059 638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:05.452446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.7.4.188:34515)
0811 06:29:05.452685 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.452690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.452878 (+ 188us) server_negotiation.cc:407] Connection header received
0811 06:29:05.453000 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.453003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.453073 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.453159 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:05.453834 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34284: BlockingRecv error: recv got EOF from 127.0.0.1:34284 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 06:29:05.540455 606 catalog_manager.cc:1259] Loaded cluster ID: 1312a6697f4940cfab357a7fa7ac9de5
I20260811 06:29:05.540557 606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: loading cluster ID for follower catalog manager: success
I20260811 06:29:05.543733 606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: acquiring CA information for follower catalog manager: success
I20260811 06:29:05.545569 606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1029823f8bba421eb65fc07fcc032f2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:29:05.593739 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.586290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.7.4.189:33421)
0811 06:29:05.586516 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.586525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.587094 (+ 569us) server_negotiation.cc:407] Connection header received
0811 06:29:05.587400 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.587406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.587516 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.587603 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.588019 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.589372 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.591509 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.592156 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.592286 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.592809 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.592942 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.593066 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.593244 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.593338 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:29:05.605940 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.600139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.7.4.188:34515)
0811 06:29:05.600381 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.600387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.600661 (+ 274us) server_negotiation.cc:407] Connection header received
0811 06:29:05.600908 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.600913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.601120 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.601231 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.601811 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.602585 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.603858 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.604286 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.604378 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.604856 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.605411 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.605522 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.605652 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.605720 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:29:05.619587 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.611922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.7.4.188:34515)
0811 06:29:05.612644 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.612652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.613184 (+ 532us) server_negotiation.cc:407] Connection header received
0811 06:29:05.613389 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.613395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.613504 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.613598 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.614486 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.615580 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.617492 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.618072 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.618187 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.618773 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.618913 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.618999 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.619223 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.619341 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553}
I20260811 06:29:05.620702 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.611770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.7.4.190:46391)
0811 06:29:05.613649 (+ 1879us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.613677 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.613697 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:29:05.613820 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.613826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.614066 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.614260 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.615092 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.616214 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.618217 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.618818 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.618990 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.619518 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.620010 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.620092 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.620293 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.620386 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1786,"thread_start_us":1961,"threads_started":1}
I20260811 06:29:05.620723 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:05.612701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.7.4.189:33421)
0811 06:29:05.612887 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:05.612893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:05.613700 (+ 807us) server_negotiation.cc:407] Connection header received
0811 06:29:05.613864 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:05.613868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:05.613950 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:05.614043 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:05.614798 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.615563 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.617731 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:05.618364 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:05.618489 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:05.618979 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:05.619097 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:05.619184 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:05.620343 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0811 06:29:05.620416 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:29:05.625186 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 325
I20260811 06:29:05.626461 391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:05.737471 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 325
I20260811 06:29:05.762975 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 404
I20260811 06:29:05.764101 477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:05.871245 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 404
I20260811 06:29:05.896153 7186 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task5M28Km/build/asan/bin/kudu pid 494
I20260811 06:29:05.897354 573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:06.004904 7186 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5M28Km/build/asan/bin/kudu with pid 494
2026-08-11T06:29:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4615 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9271 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (372025 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST